Medicine display position intelligent distribution method and system based on computer vision

Abstract

The invention provides a medicine display position intelligent distribution method and system based on computer vision, and relates to the technical field of computer vision, and the method comprises the steps: collecting a drugstore real-time image sequence through a camera of a target drugstore, carrying out the goods shelf level recognition and medicine recognition, and constructing a medicine layout model; segmenting a user image sequence from a drugstore real-time image sequence to perform complexity calculation of a medicine taking path, and establishing complexity marking information; performing medicine taking behavior pattern analysis on any user according to the user image sequence to generate medicine taking behavior mark information; the complexity marking information and the medicine taking behavior marking information are added into a cloud medicine marking information base, the medicine layout model is optimized under the constraint of medicine display specifications, and distribution optimization of medicine display positions is completed. The technical problem that the medicine taking efficiency is affected due to the fact that the medicine display optimization result and the actual medicine purchasing process are disjointed in the prior art can be solved, and the technical effect of improving the medicine taking efficiency is achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, in particular to a medicine display position intelligent allocation method and system based on computer vision. BACKGROUND

[0002] With the continuous expansion of the scale of pharmacies and the increasing variety of medicines, medicine display and management have gradually become an important link affecting customer medicine purchasing experience and pharmacy operation efficiency. At present, the existing medicine display optimization research mostly focuses on static modeling and classified placement, such as regional management according to the dosage form, use or sales frequency of medicines, but lacks in-depth analysis of real customer behavior paths. Therefore, the existing technology often only realizes layout optimization in theory, but does not combine the actual medicine taking path complexity and behavior patterns of customers, so the effect is limited.

[0003] In summary, in the prior art, due to the lack of dynamic identification and analysis of the real medicine taking path complexity and behavior patterns of customers, the medicine display optimization result is out of touch with the actual medicine purchasing process, which further affects the overall effect of the pharmacy in improving customer medicine taking efficiency, reducing search confusion and mistaken touch of medicines. SUMMARY

[0004] The purpose of the present application is to provide a medicine display position intelligent allocation method and system based on computer vision, to solve the technical problem in the prior art that due to the lack of dynamic identification and analysis of the real medicine taking path complexity and behavior patterns of customers, the medicine display optimization result is out of touch with the actual medicine purchasing process, which further affects the overall effect of the pharmacy in improving customer medicine taking efficiency, reducing search confusion and mistaken touch of medicines.

[0005] In view of the above problems, the present application provides a medicine display position intelligent allocation method and system based on computer vision.

[0006] In a first aspect, the present application provides a medicine display position intelligent allocation method based on computer vision, which is realized by a medicine display position intelligent allocation system based on computer vision, comprising: collecting real-time image sequences of a pharmacy by a camera of the target pharmacy, performing shelf level recognition and medicine recognition, and constructing a medicine layout model; calculating the complexity of the medicine taking path from the user image sequence of any user from entering the pharmacy to taking a target medicine from the real-time image sequences of the pharmacy, and establishing complexity labeling information of the target medicine; analyzing the medicine taking behavior patterns of the any user according to the user image sequence, and generating medicine taking behavior labeling information of the target medicine; adding the complexity labeling information and the medicine taking behavior labeling information of the target medicine into a cloud medicine labeling information library, optimizing the medicine layout model under the constraint of medicine display specifications, and completing the allocation optimization of the medicine display position.

[0007] Preferably, the computer vision-based intelligent dispensing method of medicine display positions further comprises: performing path analysis according to the user image sequence to construct a medicine taking path; and performing path length, inflection point and fluency identification on the medicine taking path, and performing complexity mapping conversion to generate complexity marking information of the target medicine.

[0008] Preferably, the computer vision-based intelligent dispensing method of medicine display positions further comprises: identifying the starting point and the ending point in the medicine taking path based on the medicine layout model, performing shortest path planning, identifying the length and the increase proportion of the inflection point of the medicine taking path relative to the shortest path and weighting, and generating a first complexity weighting coefficient; identifying the path fluency characteristics of the user at each path node in the medicine taking path according to the user image sequence, performing ratio calculation with the preset fluency characteristics, and generating a second complexity weighting coefficient; performing basic complexity identification based on the medicine layout model to obtain the basic path complexity of the target medicine; and performing complexity weighting on the basic path complexity of the target medicine by using the first complexity weighting coefficient and the second complexity weighting coefficient to generate the complexity marking information of the target medicine.

[0009] Preferably, the computer vision-based intelligent dispensing method of medicine display positions further comprises: the path fluency characteristics include the staying time of each path node and the number of backtracking paths.

[0010] Preferably, the computer vision-based intelligent dispensing method of medicine display positions further comprises: performing shortest medicine taking path planning on medicines at any position based on the medicine layout model to generate shortest paths of each medicine; performing mean path identification based on path length and inflection point on the shortest paths of each medicine, setting the basic path complexity of the mean path as a complexity median value; identifying the change proportion of the path length and the inflection point of the remaining medicine paths relative to the mean path, and performing complexity proportional mapping based on the complexity median value according to the change proportion to generate the basic path complexity of the remaining medicine paths; generating each basic path complexity by using the basic path complexity of the mean path and the basic path complexity of the remaining medicine paths; and identifying the target medicine according to the user image sequence, and extracting the basic path complexity of the target medicine from the each basic path complexity.

[0011] Preferably, the computer vision-based intelligent dispensing method of medicine display positions further comprises: identifying user behavior characteristics of each node of the medicine taking path from the user image sequence; constructing a preset feature of a searching-confusion mode, comparing the user behavior characteristics of each node with the preset feature of the searching-confusion mode, identifying a set of mistaken touch medicines corresponding to the nodes with the searching-confusion mode, and establishing the medicine taking behavior marking information by using the nodes with the searching-confusion mode and the set of mistaken touch medicines.

[0012] Preferably, the computer vision-based intelligent dispensing method of medicine display positions further comprises: identifying, in the cloud-based medicine mark information library, a first medicine and a first medicine display position whose recognition complexity is greater than a preset complexity and whose mark frequency is greater than a preset frequency, and a low-frequency medicine position set whose mark behavior is lower than a preset threshold; under the constraint of the medicine display specification, performing position adjustment analysis based on the medicine layout model, taking the first medicine and the first medicine display position as adjustment objects, taking the low-frequency medicine position set as an optimization space, and taking reducing path complexity as a target to obtain an optimization result; identifying, in the cloud-based medicine mark information library, whether the first medicine has a medicine taking behavior mark information, and if so, sending a mistaken touch reminding mark corresponding to a mistaken touch medicine set to a terminal management user for mark reminding.

[0013] Preferably, the computer vision-based intelligent dispensing method of medicine display positions further comprises that the constraint of the medicine display specification comprises a hard layout condition of the medicine layout.

[0014] Preferably, the computer vision-based intelligent dispensing method of medicine display positions further comprises: performing position adjustment analysis with the target of reducing path complexity, wherein the adjustment mode comprises collective regulation of similar medicines based on dosage forms or uses and independent regulation of single medicines, and if the independent regulation of single medicines, establishing a single medicine partition mark.

[0015] In a second aspect, the present application further provides a computer vision-based intelligent dispensing system of medicine display positions, which is used to execute the computer vision-based intelligent dispensing method of medicine display positions as described in the first aspect, and comprises: a medicine layout model construction module, which is used to collect real-time image sequences of a target pharmacy by using a camera of the pharmacy, perform shelf level recognition and medicine recognition, and construct a medicine layout model; a complexity mark information establishment module, which is used to calculate the complexity of a medicine taking path from entering the pharmacy to taking a target medicine by any user based on a user image sequence segmented from the real-time image sequences of the pharmacy, and establish complexity mark information of the target medicine; a medicine taking behavior mark information generation module, which is used to analyze a medicine taking behavior mode of the any user according to the user image sequence, and generate medicine taking behavior mark information of the target medicine; and a dispensing optimization module, which is used to add the complexity mark information and the medicine taking behavior mark information of the target medicine into a cloud-based medicine mark information library, optimize the medicine layout model under the constraint of a medicine display specification, and complete dispensing optimization of medicine display positions.

[0016] The technical solutions provided in the application have at least the following technical effects or advantages: by achieving the technical target of computer vision-based dynamic optimization and intelligent distribution of medicine layout, the technical effects of shortening the customer medicine taking path, improving the medicine purchase fluency, reducing the mis-touch rate, and enhancing the overall operation intelligent level of the pharmacy are achieved.

[0017] The above description is only a summary of the technical solutions of the application. In order to more clearly understand the technical means of the application, the following detailed embodiments of the application are described in accordance with the content of the description, and in order to make the above and other purposes, characteristics and advantages of the application more obvious and easy to understand, the following detailed embodiments of the application are described. It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the application, nor is it intended to limit the scope of the application. Other features of the application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are only exemplary, and for those skilled in the art, other drawings can be obtained without creative labor on the basis of the provided drawings.

[0019] Figure 1 The flowchart of the computer vision-based intelligent distribution method of medicine display position of the application.

[0020] Figure 2 The structure diagram of the computer vision-based intelligent distribution system of medicine display position of the application.

[0021] Explanation of reference signs: medicine layout model construction module 1, complexity marking information establishment module 2, medicine taking behavior marking information generation module 3, distribution optimization module 4. DETAILED DESCRIPTION

[0022] The application provides a computer vision-based intelligent distribution method and system of medicine display position, which solves the technical problem in the prior art that due to the lack of dynamic recognition and analysis of the real medicine taking path complexity and behavior mode of customers, the medicine display optimization result is disconnected with the actual medicine purchase process, which further affects the overall effect of the pharmacy in improving the customer medicine taking efficiency, reducing the confusion in finding and the behavior of mis-touching medicine. The technical target of computer vision-based dynamic optimization and intelligent distribution of medicine layout is achieved, and the technical effects of shortening the customer medicine taking path, improving the medicine purchase fluency, reducing the mis-touch rate, and enhancing the overall operation intelligent level of the pharmacy are achieved.

[0023] Below, the technical solutions in the present application will be described clearly and completely with reference to the drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. It should be understood that the present application is not limited by the example embodiments described herein. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application. In addition, it should be noted that, for the convenience of description, only part of the present application is shown in the drawings, not all.

[0024] Embodiment one, please refer to the attached Figure 1 The present application provides a computer vision-based intelligent allocation method for medicine display positions, which is applied to a computer vision-based intelligent allocation system for medicine display positions, and specifically includes the following steps: S1: Collecting real-time image sequences of a target pharmacy by using a camera of the target pharmacy, performing shelf level recognition and medicine recognition, and constructing a medicine layout model.

[0025] Specifically, the target pharmacy refers to a pharmacy that needs to be optimized for medicine display. The camera is a video acquisition device installed in the target pharmacy for continuously acquiring live pictures. The real-time image sequences are image pictures continuously acquired by the camera in chronological order. The shelf level recognition refers to identifying the hierarchical structure of the target pharmacy shelves in the real-time image sequences by computer vision technology, such as the first layer, the second layer, etc., for clearly defining the physical location of medicine display. The medicine recognition is to identify the medicine packaging in the real-time image sequences by using image recognition algorithms, so as to distinguish different medicine categories and specifications. According to the results of shelf level recognition and medicine recognition, the medicine layout model is constructed to form a virtual layout that can be represented in the computer, which is used to reflect the real medicine display situation inside the pharmacy.

[0026] S2: Calculating the complexity of the medicine taking path from entering the pharmacy to taking the target medicine by any user from the user image sequences segmented from the real-time image sequences of the pharmacy, and establishing complexity label information of the target medicine.

[0027] Specifically, the user image sequence of any user from entering the pharmacy to taking the target medicine is segmented from the real-time image sequence of the pharmacy, which is continuously collected by the camera installed in the pharmacy, and the complete process of a certain user from entering the pharmacy to taking the target medicine is extracted using the obtained images. The user image sequence refers to the continuous images recorded when the user is active in the pharmacy. The complexity of the medicine taking path is calculated, that is, the walking route of the user in the extracted user image sequence is analyzed, including path length, number of turns, dwell time, and whether there is a backtracking, etc. factors to measure the difficulty of the medicine taking process. For example, the longer the path, the more turns, the longer the stay, the higher the complexity, and the higher the value of the complexity marking information, and vice versa.

[0028] S3: performing medicine taking behavior pattern analysis on the user image sequence of any user to generate the medicine taking behavior marking information of the target medicine.

[0029] Specifically, the medicine taking behavior pattern analysis of any user is performed according to the user image sequence, that is, the complete process of a single user from entering the pharmacy to taking the medicine is extracted from the continuous video data captured by the camera, and the action and behavior characteristics of the user at different stages are identified using computer vision algorithms. The medicine taking behavior pattern analysis is to summarize the user characteristics, such as whether the user quickly finds the medicine, whether the user stays in front of the shelf for too long, whether the user appears to wander back and forth, whether the user accidentally touches other medicines, etc. to reveal the habits and difficulties of the user when searching for and taking medicine. The behavior characteristics exhibited by the user when taking a certain medicine are recorded in the form of data tags and are bound to the medicine to form a long-term accumulated behavior information library, and the medicine taking behavior marking information of the target medicine is generated.

[0030] S4: adding the complexity marking information and the medicine taking behavior marking information of the target medicine into the cloud medicine marking information library, optimizing the medicine layout model under the constraint of the medicine display specification, and completing the allocation optimization of the medicine display position.

[0031] Specifically, the complexity marking information and the medicine taking behavior marking information of the target medicine are added into the cloud medicine marking information library, that is, the path complexity and user behavior characteristic data of each medicine obtained are uploaded to the centrally managed cloud database. The complexity marking information records the path difficulty of the user in the medicine taking process, and the medicine taking behavior marking information records the confusion, accidental touch, or stay of the user in the medicine taking process. The cloud medicine marking information library is a system for centralized storage and management of information, which can accumulate data across stores and over time to achieve long-term analysis and statistics. By storing the data in the cloud medicine marking information library, a large number of user behaviors can be summarized to form a global view of the medicine taking difficulty and behavior pattern.

[0032] The drug layout model is optimized under the constraints of drug display specifications. That is, while ensuring that the pharmacy display complies with relevant regulations and standards, the specific placement of drugs is adjusted using drug labeling information. Drug display specification constraints include rigid conditions for placing drugs according to category, dosage form, use, or safety requirements. For example, prescription drugs and over-the-counter drugs cannot be mixed, and children's medicines must be placed at a height that children can reach. The drug layout model is a virtual model of the relationship between shelf location, drug location, and space. Through optimization, the complexity of the user's medication collection path can be reduced, the efficiency of medication collection can be improved, and at the same time, accidental touches and confusing behaviors can be reduced. Completing the optimization of the allocation of drug display locations means adjusting the position of various types of drugs on the shelves based on the analysis results to make the overall medication collection process smoother.

[0033] Furthermore, the present application also includes: performing path analysis based on the user image sequence to construct a medicine-picking path; performing path length, inflection point and smoothness identification on the medicine-picking path, performing complexity mapping conversion, and generating complexity marking information of the target medicine.

[0034] Specifically, path analysis is performed based on user image sequences. This involves using continuous camera footage to track the user's movements from entering the pharmacy to ultimately picking up medications, thereby constructing a medication-collection path. Path analysis uses user image sequences to calculate the user's walking route and movement sequence. The medication-collection path is the complete route from the user entering the store, passing by the shelves, stopping, selecting, and finally receiving the desired medication.

[0035] Next, the path length, inflection points, and smoothness of the medication collection path are identified. Path length refers to the total distance the user travels, inflection points are the number of turns or changes of direction along the path, and smoothness indicates whether the user pauses, backtracks, or walks back and forth along the path. Complexity mapping transforms path length, inflection points, and smoothness features into a numerical model that represents the difficulty of the path, thereby obtaining complexity marker information that indicates the difficulty of obtaining the target medication. Table 1 shows the complexity marker table for the target medication collection path.

[0036] Table 1: Target drug collection path complexity marking table

[0037] Further, the application further comprises: based on the medicine layout model, identifying the starting point and the ending point in the medicine taking path, performing shortest path planning, identifying the length and the increase proportion of inflection points of the medicine taking path relative to the shortest path and performing weighting, generating a first complex weighting coefficient; identifying the path fluency characteristics of the user at each path node in the medicine taking path according to the user image sequence, performing ratio calculation with the preset fluency characteristics, and generating a second complex weighting coefficient; performing basic complexity identification based on the medicine layout model, and obtaining the basic path complexity of the target medicine; and performing complex weighting on the basic path complexity of the target medicine by using the first complex weighting coefficient and the second complex weighting coefficient, and generating the complexity marking information of the target medicine.

[0038] Specifically, the starting point and the ending point in the medicine taking path are identified based on the medicine layout model, that is, the starting point when the user enters the pharmacy and the ending point when the target medicine position is taken are determined through the medicine layout model of the medicine position and the shelf structure. The shortest path distance and the turning are calculated according to the starting point and the ending point, which are used as a reference for comparison. Then, the increase proportion of the actual medicine taking path relative to the shortest path in length and inflection points is identified. After weighting calculation of the increase proportion of the shortest path length and the inflection points, the first complex weighting coefficient reflecting the path complexity is obtained.

[0039] The path fluency characteristics of the user at each path node in the medicine taking path are identified according to the user image sequence. The behavior of the user at different positions in the shortest path is recorded by the camera, such as the staying time, the number of backtracking and whether there is wandering. The preset fluency characteristics are ideal fluency behavior reference values defined in advance, for example, the average staying time of the user is less than 3 seconds and there is almost no backtracking under normal circumstances. The smoothness of the user's path is quantified by ratio calculation of the actual path fluency characteristics and the preset fluency characteristics. The ratio result is converted into the second complex weighting coefficient, which is used to represent the complexity of the actual behavior of the user.

[0040] The basic complexity identification is performed based on the medicine layout model, that is, the individual differences of the user are not considered, and only the positions of the medicines in the shelves are relied on for calculation, so as to obtain the medicine taking complexity of the target medicine under average circumstances. For example, the shortest path length and the number of turns are calculated for all medicines, and then the results are mapped into complexity values to obtain the basic path complexity of the target medicine.

[0041] Finally, the basic path complexity of the target medicine is complex weighted by using the first complex weighting coefficient and the second complex weighting coefficient, that is, the coefficient based on the path deviation degree and the coefficient based on the user fluency behavior jointly act on the basic complexity, so as to obtain the complexity marking information more in line with the actual user experience.

[0042] Further, the application further includes that the path fluency feature includes a stay duration of each path node and a backtracking path distribution quantity.

[0043] Specifically, the path fluency feature refers to a data index used to measure whether the user walks smoothly in the process of taking medicine. The ease of use of the path is reflected by analyzing the behavior of the user at different positions in the path. The path node is a key point divided on the user's walking path, such as a shelf intersection, a corner position, or in front of the shelf where the medicine is located. The path node is a place where the user may stop, turn or backtrack. The stay duration refers to the time the user stops at a certain path node, which is calculated in seconds. If the stay time is too long, it means that the user may be looking for medicine or confused, such as the user may not find it, resulting in an unsmooth path and a long time to take medicine. The backtracking path distribution quantity refers to the number of times the user walks back in the path and the distribution of backtracking at different path nodes. If the backtracking occurs frequently and is distributed in multiple nodes, it means that the process of taking medicine is not smooth.

[0044] Further, the application further includes: based on the medicine layout model, performing shortest medicine taking path planning on the medicine at any position to generate each medicine shortest path; performing mean path recognition based on path length and inflection point on the each medicine shortest path, setting the base path complexity of the mean path as the complexity median value; identifying the change proportion of the path length and the inflection point of the remaining medicine path relative to the mean path, and performing complexity proportional mapping based on the complexity median value according to the change proportion to generate the base path complexity of the remaining medicine path; generating each base path complexity based on the base path complexity of the mean path and the base path complexity of the remaining medicine path; performing target medicine recognition according to the user image sequence, and extracting the base path complexity of the target medicine from the each base path complexity.

[0045] Specifically, based on the medicine layout model, the shortest medicine taking path planning is performed on the medicine at any position to generate each medicine shortest path, that is, the established medicine layout model is used to calculate the optimal walking route required from the entrance position to each medicine in the drugstore. The shortest medicine taking path planning refers to finding a path with the shortest distance and the least number of turns through an algorithm. After calculating all medicines in the drugstore, a set of optimal routes is formed to obtain each medicine shortest path.

[0046] Then, the mean path based on path length and inflection point is identified for each shortest path of the medicine, and the length and the number of inflection points of all the shortest paths of the medicine are averaged to obtain a mean path representing the overall level. The base path complexity of the mean path is set as the complexity median, that is, the difficulty index when the individual behavior of the user is not considered as the complexity median. The complexity median is a reference value for calibrating the complexity. Among them, the complexity of the path with the path length and the number of inflection points in the mean range is in the middle, that is, 0.5, and then other paths are compared with the mean path, and the value of 0 to 1 is assigned according to the increase or decrease.

[0047] Subsequently, the change ratio of the path length and the inflection point of the remaining medicine path relative to the mean path is identified, and the complexity is mapped in the same ratio based on the complexity median according to the change ratio, that is, each medicine path is compared with the mean path, the increase / decrease ratio of the length and the inflection point is calculated, and then the complexity value is adjusted synchronously according to the increase / decrease ratio to generate the base complexity of the remaining medicine path.

[0048] Then, the base path complexity of each medicine is generated based on the base path complexity of the mean path and the base path complexity of the remaining medicine path, indicating that the complexity calculation results of all medicines are unified and sorted to obtain a complete base complexity list, which contains the path complexity distribution of all medicines in the pharmacy.

[0049] Finally, the target medicine is identified according to the user image sequence, and the base path complexity of the target medicine is extracted from each base path complexity, that is, when the user is searching for a specific medicine, the base path complexity of the medicine can be directly called from the list.

[0050] Further, the application also includes: identifying the user behavior features of each node of the medicine taking path from the user image sequence; constructing a preset feature of the search-confusion mode, comparing the user behavior features of each node with the preset feature of the search-confusion mode, identifying the mis-touch medicine set corresponding to the node with the search-confusion mode, and establishing the medicine taking behavior marking information based on the node with the search-confusion mode and the mis-touch medicine set.

[0051] Specifically, the user behavior features of each node of the medicine taking path are identified from the user image sequence, that is, in the complete user action trajectory recorded by the camera, the path is divided into several key nodes, such as entering the shelf area, stopping in front of a certain shelf, reaching out to take medicine, etc., and the behavior of each node is identified to obtain user behavior features, such as stay time, hand movement, gaze direction, and whether to turn back or wander, etc.

[0052] Subsequently, preset features of the search-confusion mode are constructed to define a behavior mode for determining whether the user is confused in the process of searching for the medicine. The search-confusion mode is manifested as that the user stays in front of the shelf for a long time, frequently turns back, repeatedly stretches hands to different medicines, or switches between different shelves. By comparing the preset features with the actual user behavior features, the nodes exhibiting the search-confusion mode can be identified, and a set of mistaken medicines is obtained, i.e., the non-target medicines contacted by the user in the confused state, for example, the user originally wants to buy a cough medicine, but repeatedly picks up anti-inflammatory medicines and cold medicines in front of the shelf, and then is recorded in the mistaken set.

[0053] Finally, the medicine-taking behavior marking information is established based on the nodes exhibiting the search-confusion mode and the set of mistaken medicines, i.e., the confused state exhibited by the user and the medicine information mistaken by the user are sorted and stored. The medicine-taking behavior marking information indicates that a certain medicine is prone to cause confusion of the user, and also indicates similar medicines prone to be mistaken, thereby providing a basis for optimizing the arrangement order and reducing the confusion of customers.

[0054] Further, the application also includes: identifying, in the cloud medicine marking information library, a first medicine and a first medicine arrangement position whose complexity is greater than a preset complexity and whose marking times are greater than a preset number of times, and a low-frequency medicine position set whose marking behavior is lower than a preset threshold; under the constraint of the medicine arrangement specification, based on the medicine layout model, taking the first medicine and the first medicine arrangement position as adjustment objects, taking the low-frequency medicine position set as an optimization space, and taking reducing the path complexity as a target, performing position adjustment analysis to obtain an optimization result; identifying, in the cloud medicine marking information library, whether the first medicine has medicine-taking behavior marking information, and if so, constructing a mistaken medicine reminding mark based on the corresponding mistaken medicine set and sending the mistaken medicine reminding mark to a terminal management user for marking reminding.

[0055] Specifically, the cloud medicine marking information library is used to statistically identify a first medicine and a first medicine arrangement position whose complexity is greater than a preset complexity and whose marking times are greater than a preset number of times, i.e., the complexity and behavior data of all medicines in the stored cloud medicine marking information library are statistically analyzed, when the path complexity of a certain medicine exceeds a set threshold and the complexity condition is marked multiple times, the medicine is defined as the first medicine, and the current shelf position of the medicine is referred to as the first medicine arrangement position. At the same time, the medicine positions rarely contacted by the user or having a marking number lower than a certain threshold are identified, and the positions are classified into a low-frequency medicine position set, representing a lower usage rate and an empty optimization space that can be adjusted.

[0056] Under the constraint of the medicine display specification, based on the medicine layout model, taking the first medicine and the first medicine display position as the adjustment object, taking the low-frequency medicine position set as the optimization space, and taking the reduction of path complexity as the target, position adjustment analysis is performed, that is, under the condition of ensuring that the medicine display rules are not violated, the medicine layout is recalculated and simulated, the first medicine to be optimized is moved from the originally high complexity position to the appropriate area in the low-frequency position set, so that the user's medicine taking path is shorter and the inflection point is less. The position adjustment analysis is to find an optimal solution by comparing the complexity differences that different positions may bring, output an optimization result, and can intuitively display whether the adjusted medicine position can effectively reduce the complexity of the user's medicine taking.

[0057] In the cloud medicine mark information library, it is identified whether the first medicine has a medicine taking behavior mark information, if yes, a mistaken touch reminding mark is sent to the terminal management user based on the corresponding mistaken touch medicine set for mark reminding, that is, it is checked whether the first medicine has a record related to user medicine taking behavior in the cloud medicine mark information library. If the medicine has confusion behavior or mistaken touch behavior, a mistaken touch reminding mark is generated in combination with the mistaken touch medicine set. The mistaken touch reminding mark is an automatically formed prompt signal to tell the pharmacy manager that a certain medicine may be easily confused by customers and needs special attention when adjusting the display. The terminal management user is the manager of the pharmacy, who can take measures such as adding a label, partitioning, or optimizing the placement after receiving the reminder through the system interface.

[0058] Further, the application further comprises: the medicine display specification constraint comprises a hard layout condition of the medicine layout.

[0059] Specifically, the medicine display specification constraint refers to the rules and standards that must be followed when optimizing the placement of medicines, such as those established by industry management departments or the pharmacy itself to ensure that the medicine display is both safe and compliant. The medicine display specification constraint involves the basic placement of medicines by category, dosage form, or purpose, and may also include requirements for medicine accessibility, theft prevention, and supervision of special medicines. For example, prescription drugs and over-the-counter drugs must be placed in separate zones.

[0060] The hard layout condition of the medicine layout refers to the mandatory requirements that cannot be violated in the specification. The hard conditions include that narcotic drugs must be placed in a special cabinet and locked, children's medicines must be placed on low shelves for easy access by parents, and high-risk medicines must be placed high to avoid being taken by children, etc.

[0061] Further, the application further comprises: the position adjustment analysis is performed to reduce the path complexity, wherein the adjustment mode includes collective regulation of similar medicines based on dosage form or purpose and independent regulation of single medicines, and wherein if it is independent regulation of single medicines, a single medicine partition mark is established.

[0062] Specifically, the position adjustment analysis aims to reduce the path complexity. The core purpose of adjusting the drug placement is to enable customers to get the target drug faster and more smoothly, thereby reducing the path length and unnecessary inflection points. Path complexity is an indicator of whether the customer can take the medicine smoothly. The lower the path complexity, the less time the customer spends and the higher the efficiency. Position adjustment analysis is to compare different possible placement schemes and find the scheme that can reduce complexity to the greatest extent.

[0063] The adjustment mode includes collective regulation of similar drugs based on dosage form or use. That is, if some drugs are the same in dosage form, such as tablets or capsules, or similar in use, such as cold medicine or cough medicine, multiple drugs can be adjusted simultaneously by the system as a whole and placed in the same area to facilitate customer selection. Collective regulation can enable customers to focus on one area when looking for a certain type of drug, reducing multiple path switching.

[0064] On the other hand, the adjustment mode includes independent regulation of a single drug, i.e. individual layout optimization for individual drugs. When the path complexity of a certain drug is particularly high, but the overall category of drugs is not a problem, the position of the drug can be adjusted individually. In order to ensure that the independently regulated drug is still effectively managed, a single drug subarea marker is established, i.e. an independent area or label is set to identify that the drug has been separated from the original category group and is stored individually.

[0065] In summary, the computer vision-based intelligent allocation method for drug display position provided by the present application has the following technical effects: by achieving the technical target of computer vision-based dynamic optimization and intelligent allocation of drug layout, the technical effects of shortening the customer's drug taking path, improving the drug purchase fluency, reducing the mis-touch rate and enhancing the overall operation intelligent level of the pharmacy are achieved.

[0066] Embodiment two, based on the same inventive concept as the computer vision-based intelligent allocation method for drug display position in the foregoing embodiments, the present application also provides a computer vision-based intelligent allocation system for drug display position. Please refer to the accompanying drawings Figure 2The system comprises: a medicine layout model construction module 1, configured to collect real-time image sequences of a target pharmacy by using a camera of the pharmacy, to perform shelf level recognition and medicine recognition, and to construct a medicine layout model; a complexity label information establishment module 2, configured to calculate the complexity of a medicine taking path of any user from entering the pharmacy to taking a target medicine by segmenting a user image sequence of the user from the real-time image sequences of the pharmacy, and to establish complexity label information of the target medicine; a medicine taking behavior label information generation module 3, configured to analyze the medicine taking behavior mode of the user according to the user image sequence, and to generate medicine taking behavior label information of the target medicine; and a distribution optimization module 4, configured to add the complexity label information and the medicine taking behavior label information of the target medicine into a cloud medicine label information library, to optimize the medicine layout model under the constraint of medicine display specifications, and to complete distribution optimization of a medicine display position.

[0067] Further, the computer vision-based intelligent medicine display position distribution system is further configured to: perform path analysis according to the user image sequence, and to construct a medicine taking path; perform path length, inflection point and fluency recognition on the medicine taking path, to perform complexity mapping conversion, and to generate complexity label information of the target medicine.

[0068] Further, the computer vision-based intelligent medicine display position distribution system is further configured to: identify the starting point and the ending point in the medicine taking path based on the medicine layout model, to perform shortest path planning, to identify the increase proportion of the length and the inflection point of the medicine taking path relative to the shortest path and to perform weighting, to generate a first complexity weighting coefficient; to identify the path fluency characteristics of the user at each path node in the medicine taking path according to the user image sequence, to perform ratio calculation with a preset fluency characteristic, and to generate a second complexity weighting coefficient; to perform basic complexity recognition based on the medicine layout model, to obtain the basic path complexity of the target medicine; and to perform complexity weighting on the basic path complexity of the target medicine by using the first complexity weighting coefficient and the second complexity weighting coefficient, to generate the complexity label information of the target medicine.

[0069] Further, the computer vision-based intelligent medicine display position distribution system is further configured to: the path fluency characteristics comprise the staying time of each path node and the number of backtracking path distributions.

[0070] Furthermore, the computer vision-based intelligent distribution system for drug display locations is also used to: plan the shortest drug-picking path for drugs at any location based on the drug layout model, and generate the shortest path for each drug; perform mean path identification based on path length and inflection point on the shortest paths of each drug, and set the basic path complexity of the mean path to the median complexity; identify the change ratio of the path length and inflection point of the remaining drug paths relative to the mean path, and perform complexity proportional mapping based on the median complexity according to the change ratio to generate the basic path complexity of the remaining drug paths; generate each basic path complexity based on the basic path complexity of the mean path and the basic path complexity of the remaining drug paths; identify the target drug based on the user image sequence, and extract the basic path complexity of the target drug from the each basic path complexity.

[0071] Furthermore, the computer vision-based intelligent drug display location allocation system is also used to: identify user behavior characteristics of each node of the drug-picking path from the user image sequence; construct preset characteristics of the search-confusion pattern, compare them with the user behavior characteristics of each node, and identify the set of mis-touched drugs corresponding to the nodes with the search-confusion pattern; and establish the drug-picking behavior marking information based on the nodes with the search-confusion pattern and the set of mis-touched drugs.

[0072] Furthermore, the computer vision-based intelligent allocation system for drug display locations is also used to: statistically identify the first drug and the first drug display location whose complexity is greater than a preset complexity and the number of markings is greater than a preset number in the cloud-based drug marking information library, as well as a set of low-frequency drug locations whose marking behavior is lower than a preset threshold; under the constraints of the drug display specifications, based on the drug layout model, with the first drug and the first drug display location as adjustment objects, with the low-frequency drug location set as the optimization space, and with reducing path complexity as the goal, perform position adjustment analysis to obtain an optimization result; identify in the cloud-based drug marking information library whether there is drug-picking behavior marking information for the first drug, and if so, construct a mis-touch reminder mark based on the corresponding mis-touch drug set and send it to the terminal management user for a marking reminder.

[0073] Furthermore, the computer vision-based intelligent allocation system for drug display locations is also used for: the drug display specification constraints include rigid layout conditions for drug layout.

[0074] Furthermore, the computer vision-based intelligent allocation system for drug display locations is also used to: perform position adjustment analysis with the goal of reducing path complexity, wherein the adjustment mode includes collective adjustment of similar drugs based on dosage form or use and independent adjustment of a single drug. In the case of independent adjustment of a single drug, a single drug partition mark is established.

[0075] The various embodiments described in this specification are intended to be exemplary only. The computer vision based intelligent medicine shelf location allocation method and specific examples in the foregoing embodiment one are also applicable to the computer vision based intelligent medicine shelf location allocation system of the present embodiment. Based on the foregoing detailed description of the computer vision based intelligent medicine shelf location allocation method, those skilled in the art can clearly understand the computer vision based intelligent medicine shelf location allocation system of the present embodiment. Therefore, for the sake of brevity, the computer vision based intelligent medicine shelf location allocation system of the present embodiment will not be described in detail here.

[0076] The above description of disclosed embodiments enables one of ordinary skill in the art to make and use the application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein can be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0077] Obviously, various modifications and changes can be made to the present application by those skilled in the art without departing from the spirit and scope of the present application. Thus, it is intended that the present application cover the modifications and changes as long as they come within the scope of the application and its equivalents.

Claims

1. A computer vision-based intelligent allocation method for drug display locations, characterized in that: include: Use the target pharmacy's camera to collect real-time image sequences of the pharmacy, perform shelf level recognition and drug identification, and build a drug layout model; Segmenting the image sequence of any user from entering the pharmacy to taking the target drug from the real-time image sequence of the pharmacy to calculate the complexity of the drug-taking path, and establishing complexity labeling information of the target drug; Analyzing the medication-taking behavior pattern of any user according to the user image sequence to generate medication-taking behavior marking information of the target medication; The complexity tag information and drug-taking behavior tag information of the target drug are added to the cloud-based drug tag information library, and the drug layout model is optimized under the constraints of drug display specifications to complete the allocation optimization of drug display locations.

2. The computer vision-based intelligent allocation method for drug display locations according to claim 1, characterized in that: Segmenting the complexity of a path taken by any user from entering the pharmacy to taking a target drug from the real-time image sequence of the pharmacy, and establishing complexity tag information of the target drug, including: Performing path analysis based on the user image sequence to construct a medication collection path; Path length, inflection points, and smoothness are identified for the medication path, and complexity mapping conversion is performed to generate complexity tag information for the target medication.

3. The computer vision-based intelligent allocation method for drug display locations according to claim 2, characterized in that: Execute path length, inflection point and smoothness identification on the medication path, perform complexity mapping conversion, and generate complexity tag information of the target drug, including: Based on the drug layout model, identifying the starting point and the end point of the drug picking path, performing shortest path planning, identifying and weighting the increase ratio of the drug picking path relative to the length and inflection point of the shortest path, and generating a first complex weighting coefficient; Identifying the path smoothness features of the user at each path node in the medication picking path based on the user image sequence, performing ratio calculation with the preset smoothness features, and generating a second complex weighting coefficient; Performing basic complexity identification based on the drug layout model to obtain the basic path complexity of the target drug; Complex weighting is performed on the basic path complexity of the target drug using the first complex weighting coefficient and the second complex weighting coefficient to generate complexity marking information of the target drug.

4. The computer vision-based intelligent allocation method for drug display locations as described in claim 3, wherein the path smoothness characteristics include the stay time of each path node and the number of fallback path distributions.

5. The computer vision-based intelligent allocation method for drug display locations according to claim 3, characterized in that: Based on the drug layout model, basic complexity identification is performed to obtain the basic path complexity of the target drug, including: Based on the drug layout model, the shortest route for taking drugs at any location is planned to generate the shortest route for each drug; Performing mean path identification based on path length and inflection point on the shortest paths of each drug, and setting the basic path complexity of the mean path to the median complexity; Identify the change ratio of the path length and inflection point of the remaining drug path relative to the mean path, perform complexity proportional mapping based on the complexity median according to the change ratio, and generate the basic path complexity of the remaining drug path; Generate each basic path complexity using the basic path complexity of the mean path and the basic path complexity of the remaining drug paths; Target drugs are identified according to the user image sequence, and basic path complexities of the target drugs are extracted from the respective basic path complexities.

6. The computer vision-based intelligent allocation method for drug display locations according to claim 1, characterized in that: It is characterized in that Analyzing the medication taking behavior pattern of any user according to the user image sequence to generate medication taking behavior marking information of the target medication includes: Identifying user behavior features of each node of the medication collection path from the user image sequence; Constructing preset features of the search-confusion pattern, comparing them with the user behavior features of each node, and identifying the set of mis-touched drugs corresponding to the nodes with the search-confusion pattern; The medicine-taking behavior marking information is established based on the nodes with the search-confusion mode and the set of accidentally touched medicines.

7. The computer vision-based intelligent allocation method for drug display locations according to claim 1, characterized in that: Optimizing the drug layout model under the constraints of drug display specifications to complete the allocation optimization of drug display locations, including: Counting and identifying, in the cloud-based drug marking information database, first drugs and first drug display locations with a complexity greater than a preset complexity and a marking count greater than a preset number, and a set of low-frequency drug locations with a marking behavior lower than a preset threshold; Under the constraints of the drug display specifications, based on the drug layout model, taking the first drug and the first drug display position as adjustment objects, taking the low-frequency drug position set as the optimization space, and reducing path complexity as the goal, position adjustment analysis is performed to obtain an optimization result; The cloud-based drug mark information library identifies whether the first drug has drug-taking behavior mark information. If so, a mistaken touch reminder mark is constructed based on the corresponding mistaken touch drug set and sent to the terminal management user for a mark reminder.

8. The computer vision-based intelligent allocation method for drug display locations according to claim 7, characterized in that: The drug display specification constraints include rigid layout conditions for drug layout.

9. The computer vision-based intelligent allocation method for drug display locations according to claim 7, characterized in that: Position regulation analysis is conducted with the goal of reducing path complexity. The regulation modes include collective regulation of similar drugs based on dosage form or use and independent regulation of a single drug. In the case of independent regulation of a single drug, a single drug partition mark is established.

10. The intelligent allocation system for drug display locations based on computer vision is characterized by: The steps for implementing the computer vision-based intelligent allocation method for drug display locations according to any one of claims 1 to 9 include: The drug layout model building module is used to use the target pharmacy's camera to collect real-time image sequences of the pharmacy, perform shelf level recognition and drug identification, and build a drug layout model; a complexity tag information establishment module for segmenting a user image sequence from the real-time image sequence of the pharmacy to the user image sequence of the target drug, performing complexity calculation on the drug-taking path, and establishing complexity tag information of the target drug; a medicine-taking behavior mark information generating module, configured to analyze the medicine-taking behavior pattern of any user according to the user image sequence, and generate medicine-taking behavior mark information of the target medicine; The allocation optimization module is used to add the complexity tag information and the drug-taking behavior tag information of the target drug into the cloud-based drug tag information library, optimize the drug layout model under the constraints of drug display specifications, and complete the allocation optimization of drug display locations.

Images