A mobile screen-based information display method and system

Abstract

The application relates to a mobile screen-based information display method and system, and relates to the field of science and education display technology, which comprises the following steps: collecting a current image photographed by a mobile screen in response to a trigger signal arranged on the mobile screen; identifying the current image based on an exhibition three-dimensional model stored in a preset exhibition database to obtain a current exhibition; finding corresponding exhibition information classification and classification information from a preset information database based on the current exhibition; displaying the exhibition information classification on the mobile screen, receiving a selected classification, defining the classification information corresponding to the selected classification as selected information, and displaying the selected information on the mobile screen. The application has the effects that image recognition is performed through the mobile screen, then each person can find the required information on the mobile screen, and there is no need to queue up in front of a public screen to view, so that the efficiency of exhibition display is improved.

Description

Technical Field

[0001] This invention relates to the field of science and education display technology, and in particular to an information display method and system based on a mobile screen. Background Technology

[0002] Exhibition halls, generally referred to as convention and exhibition venues, are places used for displaying goods, holding conferences and exchanges, disseminating information, and conducting economic and trade activities. Among them, science and education exhibition halls are a type of exhibition hall. In science and education exhibition halls, scientific and educational knowledge is displayed to visitors through a combination of text, images, and videos on display screens.

[0003] During exhibitions, for some difficult-to-understand scientific and educational knowledge, detailed explanations of various dimensions of professional knowledge within the field are needed when popularizing it. If text is displayed on exhibition boards, limited space often makes it difficult to convey some of the knowledge, or the large amount of information required may result in small, densely packed text, negatively impacting the visitor experience. To address the issue of conventional exhibition boards' inability to provide a detailed explanation of the overall structure of the knowledge, modern science and education venues often use touchscreen displays to further enhance the presentation of information on conventional boards. Touching the text automatically displays its definition on the screen.

[0004] While touch screens have solved the aforementioned problems, if many people want to learn about the same exhibit but the display panel can only show one item, most people will have to queue or give up, which also affects the visitor experience. Summary of the Invention

[0005] To address the issue that when a display board can only show one piece of content, most people have to queue or give up, which also affects the visitor experience, this invention provides an information display method and system based on a mobile screen.

[0006] In a first aspect, the present invention provides an information display method based on a mobile screen, which adopts the following technical solution:

[0007] A method for displaying information on a mobile screen includes:

[0008] In response to a trigger signal set on the mobile screen, the current image captured by the mobile screen is obtained;

[0009] The current exhibit is identified by recognizing the current image based on the 3D models of the exhibits stored in the pre-set exhibit database;

[0010] Based on the current exhibits, the corresponding exhibit information categories and classification information are retrieved from the preset information database;

[0011] Display the classification of the exhibit information on the mobile screen, and receive a selected classification, define the classification information corresponding to the selected classification as selected information;

[0012] Display the selected information on the mobile screen.

[0013] By using the above technical solution, the image is recognized through the mobile screen, and then each person finds the information needed by himself on his own mobile screen, without queuing up to view on a public screen, thereby improving the efficiency of exhibit display.

[0014] Optionally, the method for identifying the current image to obtain the current exhibit includes:

[0015] Determine the angle image based on the three-dimensional model of the exhibit stored in the exhibit database;

[0016] Match the angle image with the current image one by one to obtain a matching exhibit with a matching success;

[0017] When the number of matching exhibits is one, output the matching exhibit as the current exhibit;

[0018] When the number of matching exhibits is greater than or equal to two, receive the mobile screen coordinates;

[0019] Find the corresponding placement coordinates from the preset placement database based on the matching exhibit;

[0020] Calculate the exhibit distance based on the mobile screen coordinates and the placement coordinates;

[0021] Filter the smallest exhibit distance and define the corresponding matching exhibit as the target matching exhibit;

[0022] Output the target matching exhibit as the current exhibit.

[0023] By using the above technical solution, if two or more exhibits are recognized on one screen, then the selection is determined according to the position of the exhibit and the position of the person, without human judgment, making the recognition process more intelligent.

[0024] Optionally, the method for calculating the exhibit distance based on the mobile screen coordinates and the placement coordinates includes:

[0025] Receive the mobile screen coordinates while receiving the mobile screen orientation;

[0026] Filter the placement coordinates based on the mobile screen orientation, the mobile screen coordinates, and the preset mobile screen shooting angle range to obtain the orientation placement coordinates;

[0027] Calculate the exhibit distance based on the mobile screen coordinates and the orientation placement coordinates.

[0028] By adopting the technical scheme, the orientation of the camera of the mobile screen and the wide-angle of the shooting are used to screen the placement coordinates, the calculation amount and comparison amount of the placement coordinates that are impossible in principle are reduced, and the calculation efficiency of the placement coordinates is improved.

[0029] Optionally, the method further comprises a judgment method of whether to screen the minimum exhibition distance and define the corresponding matching exhibition as the target matching exhibition, the method comprising:

[0030] defining the orientation placement coordinate with the minimum exhibition distance as the minimum placement coordinate and the other orientation placement coordinates as other placement coordinates;

[0031] finding the corresponding minimum shooting area from the preset shooting database based on the minimum placement coordinate;

[0032] finding the corresponding other shooting area from the shooting database based on the other placement coordinates;

[0033] when the minimum shooting area and the other shooting area coincide, defining the other placement coordinates as the first optional placement coordinates and the corresponding matching exhibition as the first optional matching exhibition;

[0034] forming a first selection option based on the target matching exhibition and the first optional matching exhibition;

[0035] outputting the first selection option without directly outputting the target matching exhibition as the current exhibition;

[0036] when the minimum shooting area and the other shooting area do not coincide, screening the placement coordinate with the minimum exhibition distance and outputting the target matching exhibition as the current exhibition.

[0037] By adopting the technical scheme, if two exhibitions are on one exhibition stand and there is no channel in between, it is also possible that the other exhibition is on the stand, thereby generating an option for the user to select, and if the two exhibitions are not on one exhibition stand and the other exhibition has its own shooting area, only the information of the nearest exhibition is generated, thereby improving the accuracy of exhibition selection.

[0038] Optionally, when the minimum shooting area and the other shooting area do not coincide, the method of screening the placement coordinate with the minimum exhibition distance and outputting the target matching exhibition as the current exhibition comprises:

[0039] receiving other mobile screen coordinates;

[0040] determining other occupied ranges based on the other mobile screen coordinates and the preset human body range;

[0041] integrating the other occupied ranges to obtain a total occupied range;

[0042] define the corresponding other placement coordinates as second optional placement coordinates, and define the corresponding matching exhibits as second optional matching exhibits when the other shooting areas are covered by the total occupied range;

[0043] form a second selection option based on the target matching exhibit and the second optional matching exhibit;

[0044] output the second selection option without directly outputting the target matching exhibit as the current exhibit;

[0045] when there are partial areas in the other shooting areas that do not fall within the total occupied range, screen the placement coordinates with the smallest exhibit distance and output the target matching exhibit as the current exhibit.

[0046] By adopting the above technical solution, if the second exhibit position best shooting area is occupied, resulting in the current photographer has to be in the area of the first exhibit to shoot, then the second exhibit may also be the corresponding demand exhibit, so it should also be considered as the corresponding option for selection, improving the accuracy of the exhibit selection.

[0047] Optionally, the method further includes whether to define the corresponding matching exhibits as second optional matching exhibits when the other shooting areas are covered by the total occupied range, and the method includes:

[0048] define the corresponding other placement coordinates as predicted optional placement coordinates;

[0049] determine intermediate placement coordinates based on the predicted optional placement coordinates, the placement coordinates, the mobile screen orientation, the mobile screen coordinates, and the mobile screen shooting angle range;

[0050] when the intermediate placement coordinates do not exist or the intermediate placement coordinates correspond to other shooting areas covered by the total occupied range, define the matching exhibits corresponding to the predicted optional placement coordinates as second optional matching exhibits;

[0051] when the intermediate placement coordinates correspond to other shooting areas that are not completely covered by the total occupied range, do not define the matching exhibits corresponding to the predicted optional placement coordinates as second optional matching exhibits.

[0052] By adopting the above technical solution, it is also possible that there is a third exhibit between the second exhibit and the first exhibit, so it is entirely possible to shoot the second exhibit in the area of the third exhibit. Therefore, in this case, the second exhibit is unlikely to be the target exhibit of the customer, so it can be abandoned.

[0053] Optionally, the method further includes an information display method when there is no current exhibit when identifying the current image to obtain the current exhibit, and the method includes:

[0054] receiving the mobile screen coordinate and the mobile screen orientation in the absence of the current exhibit;

[0055] finding all possible display coordinates from the display database;

[0056] screen orientation, the mobile screen coordinate and the mobile screen shooting angle range to obtain the exhibit orientation display coordinate closest to the mobile screen coordinate;

[0057] finding the corresponding exhibit shooting area from the shooting database based on the exhibit orientation display coordinate;

[0058] determining the overlapping area based on the exhibit shooting area, the mobile screen orientation, the mobile screen coordinate and the mobile screen shooting angle range;

[0059] outputting the matching exhibit corresponding to the exhibit orientation display coordinate as the current exhibit when the overlapping area is completely covered by the total occupied range;

[0060] outputting a preset no-exhibit signal when the overlapping area is not completely covered by the total occupied range.

[0061] By using the above technical solution, when there is no corresponding exhibit displayed on the screen, it is determined whether the front is blocked by a person, and if so, the corresponding exhibit is directly outputted by using the coordinate, thereby improving the intelligence of the exhibit display.

[0062] Optionally, the method for determining the overlapping area based on the exhibit shooting area, the mobile screen orientation, the mobile screen coordinate and the mobile screen shooting angle range comprises:

[0063] determining the falling area based on the exhibit shooting area, the mobile screen coordinate and the mobile screen shooting angle range;

[0064] screen coordinate and the exhibit orientation display coordinate to obtain the partial falling area between the mobile screen coordinate and the exhibit orientation display coordinate, and outputting the partial falling area as the overlapping area.

[0065] By using the above technical solution, even if there is a person in the rear area, the person cannot block the front exhibit, so the display area of the front exhibit is only determined, thereby improving the efficiency of the determination.

[0066] Optionally, the method further comprises a method for finding and outputting the current exhibit in the absence of the exhibit orientation display coordinate, and the method comprises:

[0067] determining a half orientation range based on the mobile screen coordinate and the mobile screen orientation in the absence of the exhibit orientation display coordinate;

[0068] Screen coordinates, half-orientation range based on the mobile screen coordinates, deviation placement coordinates are screened to obtain the closest deviation placement coordinates;

[0069] Based on the deviation placement coordinates, the corresponding deviation shooting area is found from the shooting database;

[0070] Based on the deviation shooting area, the deviation overlap area is determined based on the mobile screen coordinates, the half-orientation range and the deviation placement coordinates;

[0071] When the deviation overlap area is completely covered by the total occupied range, the matching exhibit corresponding to the deviation placement coordinates is output as the current exhibit;

[0072] When the deviation overlap area is not completely covered by the total occupied range, a no-exhibit signal is output.

[0073] By adopting the above technical scheme, when the current side is blocked by a person, there may be no alignment of the exhibit, so at this time, whether the line of sight of the nearest exhibit is blocked is considered.

[0074] In the second aspect, the present application provides an information display system based on a mobile screen, which adopts the following technical scheme:

[0075] An information display system based on a mobile screen, comprising:

[0076] An acquisition module for acquiring a current image, mobile screen coordinates and mobile screen orientation;

[0077] A memory for storing the program of the control method of the information display method based on a mobile screen as described above;

[0078] A processor for loading and executing the program in the memory.

[0079] By adopting the above technical scheme, image recognition is performed through the mobile screen, and then each person searches for the information needed by himself on his own mobile screen, without queuing up to view on a public screen, thereby improving the efficiency of exhibit display.

[0080] In summary, the present application has at least one of the following beneficial technical effects:

[0081] 1. Image recognition is performed through the mobile screen, and then each person searches for the information needed by himself on his own mobile screen, without queuing up to view on a public screen, thereby improving the efficiency of exhibit display.

[0082] 2. If two or more exhibits are identified on a screen, then according to the position of the exhibit and the position of the person, it is determined which one is selected, without human judgment, so that the identification process is more intelligent;

[0083] 3. When the corresponding exhibit is not displayed on the screen, it is judged whether it is blocked by a person, if the front is blocked by a person, the corresponding exhibit is directly outputted by using the coordinates, and the intelligence of the exhibit display is improved. BRIEF DESCRIPTION OF DRAWINGS

[0084] Figure 1 is a flow chart of a mobile screen-based information display method in the embodiment of the present application.

[0085] Figure 2 is a flow chart of a method of identifying a current image to obtain a current exhibit in the embodiment of the present application.

[0086] Figure 3 is a flow chart of a method of calculating an exhibit distance based on a mobile screen coordinate and a placement coordinate in the embodiment of the present application.

[0087] Figure 4 is a simple schematic diagram of a mobile screen aligning an exhibit in the embodiment of the present application.

[0088] Figure 5 is a flow chart of a judgment method of whether to screen a minimum exhibit distance and define a corresponding matching exhibit as a target matching exhibit in the embodiment of the present application.

[0089] Figure 6 is a flow chart of a method of screening a placement coordinate with a minimum exhibit distance and outputting a target matching exhibit as a current exhibit when a minimum shooting area and other shooting areas do not overlap in the embodiment of the present application.

[0090] Figure 7 is a flow chart of a method of whether to define a corresponding matching exhibit as a second optional matching exhibit when other shooting areas are covered by a total occupied range in the embodiment of the present application.

[0091] Figure 8 is a flow chart of an information display method when a current exhibit does not exist when a current image is identified to obtain the current exhibit in the embodiment of the present application.

[0092] Figure 9 is a flow chart of determining an overlapping area based on a no-exhibit shooting area, a no-exhibit mobile screen orientation, a no-exhibit mobile screen coordinate and a mobile screen shooting angle range in the embodiment of the present application.

[0093] Figure 10 is a flow chart of a method of finding a current exhibit for outputting when a no-exhibit orientation placement coordinate does not exist in the embodiment of the present application. DETAILED DESCRIPTION

[0094] The present application is further described in detail below with reference to the accompanying drawings and embodiments.

[0095] The embodiment of the present application discloses a mobile screen-based information display method. Referring to Figure 1 The mobile screen-based information display method comprises the following steps.

[0096] Step 100: Collecting a current image photographed by a mobile screen in response to a trigger signal arranged on the mobile screen.

[0097] The trigger signal is a signal triggering the mobile screen to start executing the information display method. The triggering action can be that a user presses a corresponding button on the mobile screen. Here, the mobile screen can be a smart tablet, such as an IPAD.

[0098] The current image is an image photographed by a camera of the mobile screen. Here, the mobile screen is arranged with a high-definition camera. The collection mode is that the camera photographs.

[0099] When the user presses the button, the system opens the camera to photograph in response to the trigger signal corresponding to the button, and obtains the current image.

[0100] Step 101: Identifying the current image based on a three-dimensional model of an exhibit stored in a preset exhibit database to obtain a current exhibit.

[0101] The current exhibit is an exhibit identified in the photographed current image. The exhibit database stores a three-dimensional model of each exhibit in an exhibition hall. The three-dimensional model is obtained by a staff member performing three-dimensional drawing on each exhibit before exhibition, and the drawing mode can be any three-dimensional model software, such as SolidWorks. The identification mode can be described in subsequent steps and will not be repeated here.

[0102] Step 102: Finding corresponding exhibit information classification and classification information from a preset information database based on the current exhibit.

[0103] The exhibit information classification is a category of information about the current exhibit, for example, if the current exhibit is a mobile phone, the exhibit information classification can be: mobile phone size, mobile phone model, internal processor, camera resolution, etc. The classification information is information corresponding to the exhibit information classification. The database stores a mapping relationship among the current exhibit, the exhibit information classification, and the classification information. The mapping relationship is obtained by a staff member classifying and then recording and storing each information of the exhibit during exhibition of the exhibit. When the system receives the corresponding current exhibit, the corresponding exhibit information classification and classification information are automatically found from the database and output.

[0104] Step 103: Displaying the exhibit information classification on the mobile screen, receiving a selected classification, and defining classification information corresponding to the selected classification as selected information.

[0105] Selecting the category is for the user to select one of the categories of the exhibit information on the mobile screen. The receiving mode is that the corresponding text of the exhibit information category will be displayed on the mobile screen, and then the user clicks. It should be noted here that the mobile screen is a touch screen.

[0106] Step 104: Display the selected information on the mobile screen.

[0107] Referring to Figure 2 The method for identifying the current image to obtain the current exhibit includes:

[0108] Step 200: Determine the angle image based on the three-dimensional model of the exhibit stored in the exhibit database.

[0109] The angle image is an image obtained by shooting the three-dimensional model of the exhibit at a certain angle. Here, it can be obtained by screenshot software, that is, when the three-dimensional model of the exhibit is rotated, the three-dimensional model is screenshot by the screenshot software.

[0110] Step 201: Match the angle image one by one with the current image to obtain the matching exhibit with a matching success.

[0111] The matching exhibit is the exhibit corresponding to the angle image with a matching success. Here, the angle image is matched with the current image first, and then when the matching is successful, the corresponding three-dimensional model of the exhibit is automatically found, and then the exhibit corresponding to the three-dimensional model of the exhibit is found.

[0112] Step 202: When the number of matching exhibits is one, output the matching exhibit as the current exhibit.

[0113] The number of matching exhibits is one, which means that there is no interference from any other exhibit, so the matching exhibit can be directly output as the current exhibit.

[0114] Step 203: When the number of matching exhibits is greater than or equal to two, receive the mobile screen coordinates.

[0115] The mobile screen coordinates are the location of the mobile screen, which can be obtained by installing a positioning instrument on the mobile screen, such as a GPS positioning instrument.

[0116] When the number of matching exhibits is greater than or equal to two, it means that it may be one of the two exhibits at this time.

[0117] Step 204: Find the corresponding placement coordinates from the preset placement database based on the matching exhibit.

[0118] The placement coordinates represent the location of the matched exhibit. The database stores the mapping relationship between matched exhibits and their placement coordinates. These coordinates are obtained by staff after all exhibits have been placed and their corresponding positions recorded. When the system receives a matched exhibit, it automatically retrieves the corresponding placement coordinates from the database and outputs them.

[0119] Step 205: Calculate the distance to the exhibit based on the coordinates of the moving screen and the placement coordinates.

[0120] The exhibit distance is the distance between the moving screen and the exhibit. It is calculated using the coordinate values ​​of two coordinate systems.

[0121] Step 206: Filter the exhibits with the smallest distance and define the corresponding matching exhibits as target matching exhibits.

[0122] Step 207: Output the target matching exhibit as the current exhibit.

[0123] Normally, people will stand in front of the corresponding exhibits to take photos, so the exhibits they need are usually closest to.

[0124] Reference Figure 3 Methods for calculating exhibit distances based on mobile screen coordinates and placement coordinates include:

[0125] Step 300: Receive the moving screen coordinates and the moving screen orientation simultaneously.

[0126] like Figure 4 As shown, the rectangle represents a moving screen, and the location of its camera is its coordinate. The circles represent the corresponding exhibits, and the coordinates of the center point of the circle are its placement coordinates. The receiving screen orientation refers to the orientation of the receiving screen, specifically the orientation of the camera. Figure 4 The dotted line in the middle indicates the orientation of the mobile screen. This can be calculated by the direction of the compass installed in the mobile screen, i.e., the angle between the compass and the vertical direction of the mobile screen, and how many degrees the mobile screen is offset from due north.

[0127] Step 301: Filter the placement coordinates based on the orientation of the mobile screen, the coordinates of the mobile screen, and the preset range of mobile screen shooting angles to obtain the orientation placement coordinates.

[0128] The shooting angle range of a mobile screen refers to the range of angles captured by the camera on the mobile screen. This angle is limited due to mechanical constraints. Figure 4 As shown, the two dotted lines on the left and right represent the shooting angle range. The orientation and placement coordinates are the possible placement coordinates within the image captured by the mobile screen camera. The filtering method is as follows: Figure 4As shown, after the mobile screen coordinates are determined, the shooting range in the actual scene is drawn according to the mobile screen orientation and the mobile screen shooting angle range, and then the placement coordinates falling within the shooting range are screened.

[0129] Step 302: Calculate the exhibit distance based on the mobile screen coordinates and the orientation placement coordinates.

[0130] The calculation method here is similar to that of step 205, which will not be described here.

[0131] Referring to Figure 5 It also includes a judgment method of whether to screen the smallest exhibit distance and define the corresponding matching exhibit as the target matching exhibit, which includes:

[0132] Step 400: Define the orientation placement coordinates with the smallest exhibit distance as the minimum placement coordinates, and define the other orientation placement coordinates as other placement coordinates.

[0133] Step 401: Find the corresponding minimum shooting area from the preset shooting database based on the minimum placement coordinates.

[0134] The minimum shooting area is a suitable area for shooting the exhibit corresponding to the minimum placement coordinates, such as Figure 4 As shown, the area between the two rectangles drawn in dotted lines with the minimum placement coordinates as the center point is the minimum shooting area. The mapping relationship between the minimum placement coordinates and the minimum shooting area is stored in the database. The corresponding shooting area is reasonably set by the staff when shooting the exhibit, and the mapping relationship between the corresponding placement coordinates and the shooting area is recorded to obtain. When the system receives the minimum placement coordinates, it automatically finds the corresponding minimum shooting area from the database and outputs.

[0135] It should be noted that the shooting area is only the width of a person, that is, there will be no second person on the straight line formed by the exhibit and any person.

[0136] Step 402: Find the corresponding other shooting area from the shooting database based on the other placement coordinates.

[0137] The other shooting area is a suitable area for shooting the exhibit corresponding to the other placement coordinates. The establishment of the database is introduced in step 401, which will not be described here. When the system receives the other placement coordinates, it automatically finds the other shooting area from the database and outputs.

[0138] Step 403: When the minimum shooting area and the other shooting area coincide, define the other placement coordinates as the first optional placement coordinates, and define the corresponding matching exhibit as the first optional matching exhibit.

[0139] When the minimum shooting area overlaps with other shooting areas, it means that the user can shoot the exhibit corresponding to the minimum placement coordinate and the exhibits corresponding to other placement coordinates. Therefore, it is impossible to determine which exhibit is the target exhibit of the user.

[0140] Step 404: Form a first selection option based on the target matching exhibit and the first optional matching exhibit.

[0141] The first selection option is an option that can be selected by the user because the target exhibit is not clear. The first selection option is formed in the form of a table in which the target matching exhibit and the first optional matching exhibit are listed, and then displayed on the screen of the mobile screen.

[0142] Step 405: Output the first selection option without directly outputting the target matching exhibit as the current exhibit.

[0143] Because it is not clear which exhibit is the exhibit that the user needs, the first selection option is outputted without directly outputting the target matching exhibit.

[0144] Step 406: When the minimum shooting area does not overlap with other shooting areas, filter the placement coordinate of the exhibit that is closest to the minimum and output the target matching exhibit as the current exhibit.

[0145] When the minimum shooting area does not overlap with other shooting areas, it means that the other exhibits are not reasonable shooting positions for shooting images. Instead, the other exhibits are images that are accidentally shot during the shooting process. Therefore, the target is clear, and the information that the user wants is the information of the target matching exhibit. Thus, the target matching exhibit can be outputted as the current exhibit.

[0146] Reference Figure 6 When the minimum shooting area does not overlap with other shooting areas, the method for filtering the placement coordinate of the exhibit that is closest to the minimum and outputting the target matching exhibit as the current exhibit includes the following steps.

[0147] Step 500: Receive other mobile screen coordinates.

[0148] The other mobile screen coordinates are coordinates of other mobile screens in the venue in addition to the mobile screen coordinates. The receiving method is similar to step 203, and thus is not described here.

[0149] Step 501: Determine other occupied ranges based on the other mobile screen coordinates and a preset human body range.

[0150] The human body range is a range occupied by a human body in a horizontal plane. For example, the human body is considered as a cylinder with a diameter of 50 cm. The other occupied range is a range blocked by a person holding the mobile screen corresponding to the other mobile screen coordinates. The determination method is to draw the human body range with the other mobile screen coordinates as the center point of the human body range.

[0151] Step 502: integrating based on other occupied ranges to obtain a total occupied range.

[0152] The total occupied range is an integration of the other occupied ranges.

[0153] Step 503: defining the corresponding other placement coordinates as second optional placement coordinates and the corresponding matching exhibits as second optional matching exhibits when the other shooting areas are covered by the total occupied range.

[0154] When the other shooting areas are covered by the total occupied range, it means that the other shooting areas are full of people and the current mobile screen user cannot move to the other shooting areas to shoot, so the user can only shoot the matching exhibits corresponding to the other placement coordinates at the current location, thus forming the second optional matching exhibits.

[0155] Step 504: forming a second selection option based on the target matching exhibits and the second optional matching exhibits.

[0156] The second selection option is an option that can be selected by the user due to the ambiguity of the target exhibits. The formation method is similar to step 404 and will not be repeated here.

[0157] Step 505: outputting the second selection option without directly outputting the target matching exhibits as the current exhibits.

[0158] Step 506: screening the placement coordinates with the smallest exhibit distance and outputting the target matching exhibits as the current exhibits when there are partial areas in the other shooting areas that do not fall within the total occupied range.

[0159] When there are partial areas in the other shooting areas that do not fall within the total occupied range, it means that there are areas in the other shooting areas that can shoot the exhibits corresponding to the other placement coordinates. If the user wants to shoot the exhibits corresponding to the other placement coordinates, he or she can move to the area to shoot, so the exhibits corresponding to the other placement coordinates are not the exhibits the user wants to shoot. Therefore, the placement coordinates with the smallest exhibit distance are screened and the target matching exhibits are outputted as the current exhibits.

[0160] Reference Figure 7 The method also includes a method for defining the corresponding matching exhibits as second optional matching exhibits when the other shooting areas are covered by the total occupied range, which includes:

[0161] Step 600: defining the corresponding other placement coordinates as predicted optional placement coordinates.

[0162] Here, the corresponding other placement coordinates are the areas in the other shooting areas covered by the total occupied range.

[0163] Step 601: determining the intermediate placement coordinate based on the predicted optional placement coordinate, the placement coordinate, the mobile screen orientation, the mobile screen coordinate and the mobile screen shooting angle range.

[0164] The intermediate placement coordinate is a placement coordinate between the predicted optional placement coordinate and the placement coordinate. The determination is, for example, Figure 4 As shown, first, the orientation placement coordinate is determined according to step 301, then two circles are formed with the distance between the mobile screen coordinate and the placement coordinate and the distance between the mobile screen coordinate and the predicted optional placement coordinate as the diameters of the circles, and the orientation placement coordinate falling between the two circles is the intermediate placement coordinate.

[0165] Step 602: defining the matching exhibit corresponding to the predicted optional placement coordinate as the second optional matching exhibit when the intermediate placement coordinate does not exist or the other shooting area corresponding to the intermediate placement coordinate is covered by the total occupied range.

[0166] When the intermediate placement coordinate does not exist, it means that the other shooting area corresponding to the predicted optional placement coordinate is covered by the total occupied range, and the placement coordinate is the position of the nearest exhibit, so it is reasonable to shoot from the range corresponding to the placement coordinate, and the matching exhibit corresponding to the predicted optional placement coordinate is defined as the second optional matching exhibit.

[0167] When the other shooting area corresponding to the intermediate placement coordinate is covered by the total occupied range, it means that the shooting area corresponding to the intermediate placement coordinate is also occupied by people, so the matching exhibit corresponding to the predicted optional placement coordinate is defined as the second optional matching exhibit.

[0168] Step 603: not defining the matching exhibit corresponding to the predicted optional placement coordinate as the second optional matching exhibit when the other shooting area corresponding to the intermediate placement coordinate is not completely covered by the total occupied range.

[0169] When the other shooting area corresponding to the intermediate placement coordinate is not completely covered by the total occupied range, it means that the exhibit corresponding to the predicted optional placement coordinate can be completely shot in the other shooting area corresponding to the intermediate placement coordinate, which is closer to the matching exhibit corresponding to the predicted optional placement coordinate. Therefore, the matching exhibit corresponding to the predicted optional placement coordinate is not the target exhibit of the user, and the matching exhibit corresponding to the predicted optional placement coordinate is not defined as the second optional matching exhibit.

[0170] Referring to Figure 8 It also includes an information display method when the current exhibit does not exist when the current image is identified to obtain the current exhibit, and the method includes:

[0171] Step 700: receiving the no-exhibit mobile screen coordinate and the no-exhibit mobile screen orientation when the current exhibit does not exist.

[0172] The mobile screen coordinate of the no-product is the coordinate of the mobile screen that cannot be identified from the current image. The no-product mobile screen orientation is the orientation of the mobile screen corresponding to the no-product mobile screen coordinate.

[0173] Step 701: Find all possible display coordinates from the display database.

[0174] The possible display coordinates are the coordinates of all exhibits entered in the display database.

[0175] Step 702: Screen the possible display coordinates based on the no-product mobile screen orientation, no-product mobile screen coordinate and mobile screen shooting angle range to obtain the no-product orientation display coordinate closest to the no-product mobile screen coordinate.

[0176] The no-product orientation display coordinate is the possible display coordinate closest to the no-product mobile screen coordinate. The determination method is to first determine the shooting range according to step 301, then screen the possible display coordinates falling within the range, then calculate the distance between the no-product mobile screen coordinate and the possible display coordinates falling within the range respectively, and the minimum distance is the no-product orientation display coordinate.

[0177] Step 703: Find the corresponding no-product shooting area from the shooting database based on the no-product orientation display coordinate.

[0178] The no-product shooting area is the suitable shooting area of the exhibit corresponding to the no-product orientation display coordinate. The database is established in step 401 and will not be repeated here. When the system receives the no-product orientation display coordinate, it automatically finds the no-product shooting area from the database for output.

[0179] Step 704: Determine the overlapping area based on the no-product shooting area, no-product mobile screen orientation, no-product mobile screen coordinate and mobile screen shooting angle range.

[0180] The overlapping area is the area where the no-product shooting area and the mobile screen shooting range overlap. Here, the mobile screen shooting range is first determined according to the no-product mobile screen coordinate, no-product mobile screen orientation and mobile screen shooting angle range, then the two are placed in the same coordinate system, and the coordinate overlapping area is the overlapping area.

[0181] Step 705: When the overlapping area is completely covered by the total occupied range, output the matching exhibit corresponding to the no-product orientation display coordinate as the current exhibit.

[0182] When the overlapping area is completely covered by the total occupied range, it means that although the button of the trigger signal is pressed, the exhibit in the direction of the camera shooting is completely blocked, so the determination of the exhibit cannot be made. In order to allow the user to read the information of the exhibit in the blocked condition, the matching exhibit corresponding to the no-product orientation display coordinate is directly output as the current exhibit.

[0183] Step 706: outputting a preset no-exhibit signal when the overlap region is not completely covered by the total occupied range.

[0184] The no-exhibit signal is a signal indicating that the exhibit cannot be recognized or that there is no exhibit. The output mode can be a text output mode.

[0185] When the overlap region is not completely covered by the total occupied range, it means that the user can completely capture the corresponding exhibit, even without entering the no-exhibit shooting area corresponding to the no-exhibit orientation placement coordinates. Therefore, the no-exhibit signal is output.

[0186] Referring to Figure 9 , the method for determining the overlap region based on the no-exhibit shooting area, the no-exhibit mobile screen orientation, the no-exhibit mobile screen coordinates, and the mobile screen shooting angle range comprises:

[0187] Step 800: determining a falling region based on the no-exhibit shooting area, the no-exhibit mobile screen coordinates, and the mobile screen shooting angle range.

[0188] The falling region is a region captured by the camera corresponding to the no-exhibit mobile screen coordinates. The determination method is described in step 401 and will not be repeated here.

[0189] Step 801: screening the falling region based on the no-exhibit mobile screen coordinates and the no-exhibit orientation placement coordinates to obtain a partial falling region located between the no-exhibit mobile screen coordinates and the no-exhibit orientation placement coordinates, and outputting the partial falling region as the overlap region.

[0190] The partial falling region is a region located between the no-exhibit mobile screen coordinates and the no-exhibit orientation placement coordinates. Here, a circle is formed with the distance between the no-exhibit mobile screen coordinates and the no-exhibit orientation placement coordinates as the diameter, and the region falling within the circle is the partial falling region.

[0191] In the region behind the no-exhibit orientation placement coordinates, even if there is a person, he or she cannot block the exhibit in front. Therefore, only the display region of the exhibit in front is judged, which improves the efficiency of the judgment.

[0192] Referring to Figure 10 , the method further comprises finding a current exhibit for output if the no-exhibit orientation placement coordinates do not exist, and the method comprises:

[0193] Step 900: determining a half-orientation range based on the no-exhibit mobile screen coordinates and the no-exhibit mobile screen orientation if the no-exhibit orientation placement coordinates do not exist.

[0194] The half-orientation range is a range of angles corresponding to a half circle of the no-exhibit mobile screen orientation. Here, a ray corresponding to the no-exhibit mobile screen orientation is determined based on the no-exhibit mobile screen coordinates, and then the angles of 90° clockwise and counterclockwise are integrated to obtain the half-orientation range.

[0195] When there is no product orientation placement coordinate, it means that the product is not aligned at this time, but it is also possible that the product is not aligned because it is blocked by someone in front, so further analysis is needed.

[0196] Step 901: Screen possible placement coordinates based on the product movement screen coordinate, the half orientation range, to obtain a deviation placement coordinate closest to the product movement screen coordinate.

[0197] The deviation placement coordinate is the possible placement coordinate closest to the product movement screen coordinate. The screening manner is similar to step 702, and the difference is that the orientation range is different.

[0198] Step 902: Find a corresponding deviation shooting area from the shooting database based on the deviation placement coordinate.

[0199] The deviation shooting area is a suitable shooting area of the product corresponding to the deviation placement coordinate.

[0200] Step 903: Determine a deviation overlap area based on the deviation shooting area, the product movement screen coordinate, the half orientation range, and the deviation placement coordinate.

[0201] The deviation overlap area is an area where the deviation shooting area and the movement screen shooting range overlap. The determination manner is similar to step 704, and details are not repeated here.

[0202] Step 904: When the deviation overlap area is completely covered by the total occupied range, output the matching product corresponding to the deviation placement coordinate as the current product.

[0203] When the deviation overlap area is completely covered, it is completely possible that the product is blocked by someone else, resulting in no alignment. At this time, the matching product corresponding to the deviation placement coordinate is output as the current product.

[0204] In addition, when there is a product orientation placement coordinate, the manner of steps 900-904 can also be used to determine whether the matching product corresponding to the deviation placement coordinate is also the target product, thereby generating an option for the user to select.

[0205] Step 905: When the deviation overlap area is not completely covered by the total occupied range, output a no product signal.

[0206] When the deviation overlap area is not completely covered by the total occupied range, it means that the user can definitely see the matching product corresponding to the deviation placement coordinate, so the matching product corresponding to the deviation placement coordinate is excluded, and therefore a no product signal is output.

[0207] Based on the same inventive concept, the embodiment of the present application provides a mobile screen-based information display system.

[0208] A mobile screen-based information display system comprises:

[0209] An acquisition module is configured to acquire a current image, mobile screen coordinates and a mobile screen orientation;

[0210] A memory is configured to store a program of a control method of a mobile screen-based information display method;

[0211] A processor is configured to load and execute the program in the memory.

[0212] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of functional modules is taken as an example, and in actual application, the above-mentioned functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the above-described system, device and unit can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0213] The above is only a preferred embodiment of the present application, and the protection scope of the present application is not limited to the above-mentioned embodiments. Any technical solution falling within the concept of the present application shall be considered as falling within the protection scope of the present application. It should be noted that, for those skilled in the art, some improvements and refinements without departing from the principles of the present application shall also be considered as falling within the protection scope of the present application.

Claims

1. A mobile screen-based information display method, characterized by, The method comprises the following steps: collecting a current image photographed by the mobile screen in response to a trigger signal arranged on the mobile screen; identifying the current image based on a three-dimensional model of an exhibit stored in a preset exhibit database to obtain a current exhibit; finding corresponding exhibit information classification and classification information from a preset information database based on the current exhibit; displaying the exhibit information classification on the mobile screen and receiving a selected classification, and defining the classification information corresponding to the selected classification as selected information; displaying the selected information on the mobile screen; wherein the method of identifying the current image to obtain the current exhibit comprises: determining an angle image based on the three-dimensional model of the exhibit stored in the exhibit database; matching the angle image with the current image one by one to obtain a matching exhibit with a matching success; when the number of the matching exhibits is one, outputting the matching exhibit as the current exhibit; when the number of the matching exhibits is greater than or equal to two, receiving a mobile screen coordinate; finding corresponding placement coordinates from a preset placement database based on the matching exhibits; calculating an exhibit distance based on the mobile screen coordinate and the placement coordinate; screening the smallest exhibit distance and defining the corresponding matching exhibit as a target matching exhibit; outputting the target matching exhibit as the current exhibit; wherein the method of calculating the exhibit distance based on the mobile screen coordinate and the placement coordinate comprises: receiving the mobile screen coordinate while receiving a mobile screen orientation; screening the placement coordinate based on the mobile screen orientation, the mobile screen coordinate and a preset mobile screen shooting angle range to obtain an orientation placement coordinate; calculating the exhibit distance based on the mobile screen coordinate and the orientation placement coordinate; defining the orientation placement coordinate with the smallest exhibit distance as a minimum placement coordinate, and defining other orientation placement coordinates as other placement coordinates; finding a corresponding minimum shooting area from a preset shooting database based on the minimum placement coordinate; finding a corresponding other shooting area from the shooting database based on the other placement coordinates; when the minimum shooting area and the other shooting area do not overlap, the method of screening the placement coordinate with the smallest exhibit distance and outputting the target matching exhibit as the current exhibit comprises: receiving other mobile screen coordinates; determining other occupied ranges based on the other mobile screen coordinates and a preset human body range; integrating the other occupied ranges to obtain a total occupied range; when the other shooting area is covered by the total occupied range, defining the corresponding other placement coordinate as a second selectable placement coordinate, and defining the corresponding matching exhibit as a second selectable matching exhibit; forming a second selection option based on the target matching exhibit and the second selectable matching exhibit; outputting the second selection option, instead of directly outputting the target matching exhibit as the current exhibit; screening the placement coordinate with the minimum distance of the exhibits and outputting the target matching exhibit as the current exhibit when there are partial regions in the other photographing regions that do not fall within the total occupied range; the judgment method of whether to screen the minimum distance of the exhibits and define the corresponding matching exhibit as the target matching exhibit, the method comprising: defining the other placement coordinates as first optional placement coordinates and the corresponding matching exhibits as first optional matching exhibits when the minimum photographing region and the other photographing regions overlap; forming a first selection option based on the target matching exhibit and the first optional matching exhibit; outputting the first selection option without directly outputting the target matching exhibit as the current exhibit; screening the placement coordinate with the minimum distance of the exhibits and outputting the target matching exhibit as the current exhibit when the minimum photographing region and the other photographing regions do not overlap; the information display method when there is no current exhibit when the current image is recognized to obtain the current exhibit, the method comprising: receiving no-exhibit mobile screen coordinates and no-exhibit mobile screen orientation when there is no current exhibit; finding all possible placement coordinates from the placement database; screening the possible placement coordinates based on the no-exhibit mobile screen orientation, the no-exhibit mobile screen coordinates and the mobile screen photographing angle range to obtain the no-exhibit orientation placement coordinates closest to the no-exhibit mobile screen coordinates; finding the corresponding no-exhibit photographing region from the photographing database based on the no-exhibit orientation placement coordinates; determining an overlapping region based on the no-exhibit photographing region, the no-exhibit mobile screen coordinates and the mobile screen photographing angle range; outputting the matching exhibit corresponding to the no-exhibit orientation placement coordinates as the current exhibit when the overlapping region is completely covered by the total occupied range; outputting a preset no-exhibit signal when the overlapping region is not completely covered by the total occupied range; the method of finding the current exhibit for output when there is no no-exhibit orientation placement coordinates, the method comprising: determining a half-orientation range based on the no-exhibit mobile screen coordinates and the no-exhibit mobile screen orientation when there is no no-exhibit orientation placement coordinates; screening the possible placement coordinates based on the no-exhibit mobile screen coordinates and the half-orientation range to obtain the deviation placement coordinates closest to the no-exhibit mobile screen coordinates; finding the corresponding deviation photographing region from the photographing database based on the deviation placement coordinates; determining a deviation overlapping region based on the deviation photographing region, the no-exhibit mobile screen coordinates, the half-orientation range and the deviation placement coordinates; outputting the matching exhibit corresponding to the deviation placement coordinates as the current exhibit when the deviation overlapping region is completely covered by the total occupied range; outputting the no-exhibit signal when the deviation overlapping region is not completely covered by the total occupied range; Also included in the method of whether the corresponding matching exhibit is defined as a second optional matching exhibit when the other shooting area corresponding to the intermediate placement coordinate is covered by the total occupied range, the method comprises: defining the corresponding other placement coordinates as predicted optional placement coordinates; determining intermediate placement coordinates based on the predicted optional placement coordinates, the placement coordinates, the mobile screen orientation, the mobile screen coordinates and the mobile screen shooting angle range; defining the matching exhibit corresponding to the predicted optional placement coordinates as a second optional matching exhibit when the intermediate placement coordinates do not exist or the other shooting area corresponding to the intermediate placement coordinates is covered by the total occupied range; not defining the matching exhibit corresponding to the predicted optional placement coordinates as a second optional matching exhibit when the other shooting area corresponding to the intermediate placement coordinates is not completely covered by the total occupied range.

2. The mobile screen-based information display method of claim 1, wherein, The method of determining the overlapping area based on the no-product shooting area, the no-product mobile screen coordinates and the mobile screen shooting angle range comprises: determining the falling area based on the no-product shooting area, the no-product mobile screen coordinates and the mobile screen shooting angle range; screening the falling area based on the no-product mobile screen coordinates and the no-product orientation placement coordinates to obtain the partial falling area between the no-product mobile screen coordinates and the no-product orientation placement coordinates, and outputting the partial falling area as the overlapping area.

3. A mobile screen-based information display system, characterized by comprise: a receiving module for receiving a current image, mobile screen coordinates and mobile screen orientation; a memory for storing the program of the control method of the mobile screen-based information display method according to any one of claims 1 to 2; a processor for loading and executing the program in the memory.

Images