Mobile terminal store inspection method, system and equipment for audio and video cloud processing, and medium

Abstract

The invention provides a mobile terminal store inspection method, system and device for audio and video cloud processing, and a medium. The method comprises the following steps: obtaining a target inspection demand to formulate a first standard of a chain store, obtaining target store information to formulate a second standard, grading inspection items according to the first standard and the second standard, and outputting a target inspection list; splitting the target inspection list adaptive to a mobile terminal display interface into atomic inspection lists, and configuring a lookup instruction and a positioning component for the atomic inspection lists to traverse the target inspection list; and obtaining an inspection point image and an inspection audio of the target store, extracting a first inspection feature and a second inspection feature of the target store, inputting the inspection features into the industry model for fusion analysis, outputting an inspection result, and filling the atomic inspection list according to the inspection structure to fill the target inspection list.

Description

Technical Field

[0001] The present invention relates to the technical field of store inspection, and in particular to a mobile terminal store inspection method, system, equipment and medium for audio and video cloud processing. Background Art

[0002] With the development of the information age, digitalization has been integrated into all walks of life. For most traditional chain industries, it is necessary to conduct store inspections in a timely manner. Before the inspection, it is necessary to generate corresponding inspection forms so that inspectors can record and evaluate the inspection results, thereby realizing store operation, inspection and management.

[0003] At present, on-site inspection work is still in the primary digital stage, and there are still some defects such as:

[0004] (1) Inspection results are not ideal. For remote surveillance video inspections, some inspection items cannot be determined through surveillance videos and still require on-site inspections by inspectors. During on-site inspections, inspectors still need to communicate with store employees to understand the detailed store situation. Otherwise, it is easy to overlook necessary inspection items, resulting in unsatisfactory inspection results.

[0005] (2) The inspection process is inefficient. Inspectors need to fill in the inspection results on a mobile inspection form. Faced with no less than a hundred inspection items, it is easy to make mistakes and it takes a long time.

[0006] Therefore, there is an urgent need for a mobile store inspection method with audio and video cloud processing to solve the problems of low inspection efficiency and unsatisfactory inspection results in the above-mentioned chain stores. Summary of the invention

[0007] In view of the deficiencies in the prior art, the present invention proposes a mobile store inspection method, system, device and medium for audio and video cloud processing.

[0008] The first aspect of the present invention discloses a mobile terminal store inspection method for audio and video cloud processing, comprising:

[0009] Obtain the target inspection requirements to formulate the first standard for chain stores, obtain the target store information to formulate the second standard, classify the inspection items according to the first standard and the second standard, and output the target inspection form;

[0010] The target inspection list is adapted to the mobile terminal display interface and split into atomic inspection lists. The atomic inspection lists are configured with reference instructions and positioning components to traverse the target inspection list.

[0011] Obtain the checkpoint images and inspection audio of the target store, extract the first inspection features and second inspection features of the target store, input the inspection features into the industry model for fusion analysis to output the inspection results, and fill in the atomic inspection form according to the inspection results to fill in the target inspection form on the mobile terminal.

[0012] In an optional embodiment, the obtaining of target inspection requirements to formulate a first standard for chain stores and the obtaining of target store information to formulate a second standard include:

[0013] Determine the target inspection requirements based on historical inspection data and inspection update rules, and determine the standard inspection items of each cluster store based on the cluster classification of chain stores to generate the first standard;

[0014] Determine, according to the target store information, special inspection items that distinguish the target store from the cluster stores, and generate a second standard with the special inspection items;

[0015] The first standard and the second standard are configured as an inspection template and saved in a database, and the inspection template is updated in real time.

[0016] In an optional embodiment, the step of grading the inspection items according to the first standard and the second standard and outputting the target inspection list includes:

[0017] The classification model is used to perform hierarchical processing on the inspection items of the first standard and the second standard, wherein the hierarchical processing inspection items include inspection categories and inspection items, and the inspection items are classified into the inspection categories;

[0018] According to the hierarchical processing, the inspection categories and the inspection items within the inspection categories are prioritized, and a first display order template is obtained to generate a target inspection list.

[0019] In an optional embodiment, the step of adapting the target inspection list to the mobile terminal display interface and splitting it into atomic inspection lists includes:

[0020] Performing a first weight allocation on the inspection category and the inspection item in the target inspection form according to the inspection form reporting type, and allocating a sliding operation distance of the target reporting item in the target inspection form from the mobile terminal display interface according to the first weight allocation, wherein the reporting type includes an audio recognition reporting item, a visual recognition reporting item, and an offline recognition reporting item;

[0021] A second weight is added to the target inspection list assigned with a first weight based on the target store information and the inspector's experience information, and the inspection items in the target inspection list are split into atomic inspection lists based on the first weight and the second weight. The atomic inspection list is configured on the mobile display interface to assign display color and display order based on the first weight and the second weight.

[0022] In an optional embodiment, configuring the atomic inspection list with a query instruction and a positioning component to traverse the target inspection list includes:

[0023] The mobile terminal fills the first atomic inspection list in the display interface, and sets the second atomic inspection list adjacent to the first atomic inspection list in sequence according to the first weight and the second weight;

[0024] The mobile terminal configures a sliding gesture, a zooming gesture and a sensing device for the second atomic inspection sheet to view the second atomic inspection sheet adjacent to the first atomic inspection sheet, and the sensing device scrolls the display content of the display interface according to the tilt state of the mobile terminal device;

[0025] The mobile terminal obtains voice information and converts it into text information, determines the position of the second atomic inspection list through the text information, and jumps the mobile terminal display interface to the corresponding position.

[0026] In an optional embodiment, the step of acquiring the inspection point image and inspection audio of the target store, extracting the first inspection feature and the second inspection feature of the target store, and inputting the inspection features into the industry model for fusion analysis and outputting the inspection results includes:

[0027] Generate an inspection plan including a target inspection list based on the target store information, collect offline inspection information and interview voice data at the target store according to the inspection plan, and collect image data corresponding to the inspection items through visual equipment;

[0028] The voice data and the image data are uploaded to the processing system, the voice data is converted into text to obtain text data, the visual processing system uses a neural network to identify the image data and extract the first inspection feature, and the text processing system uses natural language processing to identify the text data and extract the second inspection feature;

[0029] Industry data and inspection data are used as training sets to update the industry model. The first inspection feature and the second inspection feature are input into the industry model to obtain the evaluation results of the corresponding inspection items on the atomic inspection list. The evaluation results from different feature sources are classified and sorted using the veto system, and the inspection results are output.

[0030] In an optional embodiment, filling in the atomic inspection form according to the inspection result to fill in the target inspection form of the mobile terminal includes:

[0031] For the target inspection form, the audio recognition reporting items and visual recognition reporting items in each atomic inspection form are automatically filled in on the mobile terminal according to the inspection results;

[0032] The offline identification and reporting items of the target inspection form are manually filled in by referring to the instructions and positioning components, and the target inspection results of the target store are recorded according to the target inspection form.

[0033] A second aspect of the present invention discloses a mobile terminal store inspection system for audio and video cloud processing, the system comprising:

[0034] Inspection form generation module, used to obtain the target inspection requirements to formulate the first standard of chain stores, obtain the target store information to formulate the second standard, classify the inspection items according to the first standard and the second standard, and output the target inspection form;

[0035] A mobile terminal configuration module is used to adapt the target inspection list to the mobile terminal display interface and split it into atomic inspection lists, and configure the atomic inspection list with a query instruction and a positioning component to traverse the target inspection list;

[0036] The inspection form filling module is used to obtain the inspection point images and inspection audio of the target store, extract the first inspection features and second inspection features of the target store, input the inspection features into the industry model for fusion analysis and output the inspection results, and fill in the atomic inspection form according to the inspection results to fill in the target inspection form on the mobile terminal.

[0037] In an optional embodiment, the inspection form generation module includes:

[0038] A standard generation module is used to determine the target inspection requirements based on historical inspection data and inspection update rules, and to determine the standard inspection items of each cluster store based on the cluster classification of the chain stores to generate the first standard;

[0039] Determine, according to the target store information, special inspection items that distinguish the target store from the cluster stores, and generate a second standard with the special inspection items;

[0040] The first standard and the second standard are configured as an inspection template and saved in a database, and the inspection template is updated in real time.

[0041] In an optional embodiment, the inspection form generation module includes:

[0042] An inspection list generation submodule, used for performing hierarchical processing on the inspection items of the first standard and the second standard by using a classification model, wherein the hierarchical processing inspection items include inspection categories and inspection items, and the inspection items are classified into the inspection categories;

[0043] According to the hierarchical processing, the inspection categories and the inspection items within the inspection categories are prioritized, and a first display order template is obtained to generate a target inspection list.

[0044] In an optional embodiment, the mobile terminal configuration module includes:

[0045] An inspection form splitting module is used to perform a first weight allocation on the inspection categories and inspection items in the target inspection form according to the inspection form reporting type, and allocate a sliding operation distance of the target reporting item in the target inspection form from the mobile terminal display interface according to the first weight allocation, wherein the reporting type includes an audio recognition reporting item, a visual recognition reporting item, and an offline recognition reporting item;

[0046] A second weight is added to the target inspection list assigned with a first weight based on the target store information and the inspector's experience information, and the inspection items in the target inspection list are split into atomic inspection lists based on the first weight and the second weight. The atomic inspection list is configured on the mobile display interface to assign display color and display order based on the first weight and the second weight.

[0047] In an optional embodiment, the mobile terminal configuration module includes:

[0048] A mobile terminal configuration submodule is used for the mobile terminal to fill the first atomic inspection list in the display interface, and to set the second atomic inspection list adjacent to the first atomic inspection list in sequence according to the first weight and the second weight;

[0049] The mobile terminal configures a sliding gesture, a zooming gesture and a sensing device for the second atomic inspection sheet to view the second atomic inspection sheet adjacent to the first atomic inspection sheet, and the sensing device scrolls the display content of the display interface according to the tilt state of the mobile terminal device;

[0050] The mobile terminal obtains voice information and converts it into text information, determines the position of the second atomic inspection list through the text information, and jumps the mobile terminal display interface to the corresponding position.

[0051] In an optional embodiment, the inspection form filling module includes:

[0052] The feature analysis module is used to generate an inspection plan including a target inspection list based on the target store information, collect offline inspection information and interview voice data at the target store according to the inspection plan, and collect image data corresponding to the inspection items through visual equipment;

[0053] The voice data and the image data are uploaded to the processing system, the voice data is converted into text to obtain text data, the visual processing system uses a neural network to identify the image data and extract the first inspection feature, and the text processing system uses natural language processing to identify the text data and extract the second inspection feature;

[0054] Industry data and inspection data are used as training sets to update the industry model. The first inspection feature and the second inspection feature are input into the industry model to obtain the evaluation results of the corresponding inspection items on the atomic inspection list. The evaluation results from different feature sources are classified and sorted using the veto system, and the inspection results are output.

[0055] In an optional embodiment, the inspection form filling module includes:

[0056] The inspection form filling submodule is used to automatically fill in the audio recognition filling items and visual recognition filling items in each atomic inspection form on the mobile terminal according to the inspection results of the target inspection form;

[0057] The offline identification and reporting items of the target inspection form are manually filled in by referring to the instructions and positioning components, and the target inspection results of the target store are recorded according to the target inspection form.

[0058] The third aspect of the present invention discloses a mobile store inspection device for audio and video cloud processing, comprising:

[0059] at least one processor, and

[0060] a memory communicatively connected to the at least one processor; wherein,

[0061] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the mobile store inspection method for audio and video cloud processing as described in any one of the first aspects of the present invention.

[0062] The fourth aspect of the present invention discloses a computer-readable storage medium, which stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the mobile store inspection method for audio and video cloud processing as described in any one of the items disclosed in the first aspect of the present invention.

[0063] Compared with the prior art, the present invention has the following advantages:

[0064] The present invention proposes a mobile terminal store inspection method, system, device and medium for audio and video cloud processing, wherein the method includes: obtaining the target inspection demand to formulate the first standard of chain stores, obtaining the target store information to formulate the second standard, grading the inspection items according to the first standard and the second standard, and outputting the target inspection list; adapting the target inspection list to the mobile terminal display interface and splitting it into atomic inspection lists, configuring the atomic inspection list with a query instruction and a positioning component to traverse the target inspection list; obtaining the inspection point image and inspection audio of the target store, extracting the first inspection feature and the second inspection feature of the target store, inputting the inspection feature into the industry model for fusion analysis and outputting the inspection result, and filling the atomic inspection list according to the inspection structure to fill in the target inspection list. The analysis results of the inspection items and inspection data processed by the cloud can help the inspectors to quickly complete the inspection process of the target stores, and at the same time introduce the operability of the mobile terminal display, which not only improves the inspection efficiency but also improves the accuracy of the inspection results, ensuring that the inspection results can reflect the actual store situation, especially in the field of chain store inspection, which has a high application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0066] Figure 1 It is a flow chart of the mobile terminal store inspection method for audio and video cloud processing of the present invention;

[0067] Figure 2 A schematic diagram of a mobile terminal display interface in the present invention;

[0068] Figure 3 A schematic diagram of a mobile store inspection system for audio and video cloud processing of the present invention. DETAILED DESCRIPTION

[0069] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments provided in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0070] Obviously, the drawings described below are only some examples or embodiments of the present application. For ordinary technicians in this field, the present application can also be applied to other similar scenarios based on these drawings without creative work. In addition, it can also be understood that although the efforts made in this development process may be complicated and lengthy, for ordinary technicians in this field related to the content disclosed in this application, some changes in design, manufacturing or production based on the technical content disclosed in this application are just conventional technical means, and should not be understood as insufficient content disclosed in this application.

[0071] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those of ordinary skill in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0072] Unless otherwise defined, the technical terms or scientific terms involved in this application should be understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the" and the like involved in this application do not indicate a quantitative limitation, and may represent the singular or plural. The terms "include", "comprise", "have" and any of their variations involved in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there may be three relationships, for example, "A and / or B" can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects before and after are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0073] Example 1

[0074] See also Figure 1 The embodiment of the present invention discloses a mobile terminal store inspection method for audio and video cloud processing, including:

[0075] 101. Obtain the target inspection requirements to formulate the first standard for chain stores, obtain the target store information to formulate the second standard, classify the inspection items according to the first standard and the second standard, and output the target inspection form;

[0076] It should be noted that most traditional chain industries are still in the early stages of digitalization in the arrangement of on-site inspections. Compared with the inspection of a single store or several chain stores, the inspection of chain stores within the chain industry needs to consider more inspection standards and rules and regulations. This will result in hundreds or thousands of fill-in items or check boxes on the forms required for inspections. In order to further realize digitalization, the specific content of the inspection required for each store should be determined and then the forms should be generated uniformly. Special considerations still need to be made for stores with special circumstances to ensure the accuracy and feasibility of the content of the target inspection form for feedback of inspection results.

[0077] In an optional embodiment, the obtaining of target inspection requirements to formulate a first standard for chain stores and the obtaining of target store information to formulate a second standard include:

[0078] Determine the target inspection requirements based on historical inspection data and inspection update rules, and determine the standard inspection items of each cluster store based on the cluster classification of chain stores to generate the first standard;

[0079] Determine, according to the target store information, special inspection items that distinguish the target store from the cluster stores, and generate a second standard with the special inspection items;

[0080] The first standard and the second standard are configured as an inspection template and saved in a database, and the inspection template is updated in real time.

[0081] Specifically, the chain headquarters can formulate inspection standards according to business needs, especially by extracting problems from some data in historical inspection data that are more likely to have inspection non-compliance, or by using the inspection rules updated by the chain headquarters as a reference to generate inspection standards. The content of standard inspection items usually includes but is not limited to store information, the image of the store's peripheral area, product sales standards, store hygiene, cargo storage, food safety issues and other related items. For cluster stores in different regions, such as halal areas, local customs and eating habits need to be considered. Therefore, the corresponding product sales standards, product production process supervision, cargo storage and other inspection items will need to be modified. The content required to be inspected for special stores is different from that for ordinary stores. For example, newly opened stores or regional trial stores need to adjust the inspection items of the first standard, or the sanitary blind spots of each store are different due to the different store layouts. Although the blind spots can be captured by adjusting the monitoring position, some special situations still need to be listed as inspection items to facilitate subsequent store inspectors to inspect and count the store conditions. After the first and second standards are formulated, they can be compiled into Excel files in the format required by the digital inspection system for subsequent regulation and use.

[0082] In an optional embodiment, the step of grading the inspection items according to the first standard and the second standard and outputting the target inspection list includes:

[0083] The classification model is used to perform hierarchical processing on the inspection items of the first standard and the second standard, wherein the hierarchical processing inspection items include inspection categories and inspection items, and the inspection items are classified into the inspection categories;

[0084] According to the hierarchical processing, the inspection categories and the inspection items within the inspection categories are prioritized, and a first display order template is obtained to generate a target inspection list.

[0085] Specifically, the grading model can be trained through store inspection data to obtain the importance (weight) of different inspection items for the inspection results of the target store. It can be understood that in order to facilitate inspectors to review the inspection list, they are first classified into various inspection categories according to the similarity or correlation of the inspection items. The inspection categories should be sorted so that the more important inspection categories are easier to display on the mobile display interface for inspectors to review and fill in. Similarly, inspection items belonging to the same inspection category should be sorted in the directory so that the display interface of the inspector's mobile terminal will give priority to displaying inspection items with higher inspection score weights under the same inspection category.

[0086] 102. Split the target inspection list into atomic inspection lists to adapt to the mobile terminal display interface, and configure the atomic inspection list with a query instruction and a positioning component to traverse the target inspection list;

[0087] It should be explained that the atomic inspection form is the smallest displayable form obtained by atomizing the target inspection form. In actual scenarios, inspectors need to go to the store for inspection and fill in the part that can only communicate the inspection results collected by offline inspections. The atomic inspection form can intuitively display the inspection items that need to be queried directly on the display interface of the mobile terminal, making it convenient for inspectors to check or fill in.

[0088] In an optional embodiment, the step of adapting the target inspection list to the mobile terminal display interface and splitting it into atomic inspection lists includes:

[0089] Performing a first weight allocation on the inspection category and the inspection item in the target inspection form according to the inspection form reporting type, and allocating a sliding operation distance of the target reporting item in the target inspection form from the mobile terminal display interface according to the first weight allocation, wherein the reporting type includes an audio recognition reporting item, a visual recognition reporting item, and an offline recognition reporting item;

[0090] A second weight is added to the target inspection list assigned with a first weight based on the target store information and the inspector's experience information, and the inspection items in the target inspection list are split into atomic inspection lists based on the first weight and the second weight. The atomic inspection list is configured on the mobile display interface to assign display color and display order based on the first weight and the second weight.

[0091] Specifically, Figure 2 As shown, the actual display state of the target inspection form is a large graphic that exceeds the mobile terminal display interface, wherein the atomic inspection form is displayed in the center of the mobile terminal screen and fills the entire screen, and other atomic inspection forms extend outside the screen, requiring gesture operations on the mobile terminal to view other atomic inspection forms. In order to improve the efficiency of inspectors in reviewing inspection forms, the distance between each atomic inspection form and the center screen of the display interface is determined according to the weight distribution of the inspection type and the inspection item, and different categories of inspection items can be distinguished by color, so that inspectors can quickly locate the position of the inspection items, thereby improving the efficiency and accuracy of inspection form filling.

[0092] In an optional embodiment, configuring the atomic inspection list with a query instruction and a positioning component to traverse the target inspection list includes:

[0093] The mobile terminal fills the first atomic inspection list in the display interface, and sets the second atomic inspection list adjacent to the first atomic inspection list in sequence according to the first weight and the second weight;

[0094] The mobile terminal configures a sliding gesture, a zooming gesture and a sensing device for the second atomic inspection sheet to view the second atomic inspection sheet adjacent to the first atomic inspection sheet, and the sensing device scrolls the display content of the display interface according to the tilt state of the mobile terminal device;

[0095] The mobile terminal obtains voice information and converts it into text information, determines the position of the second atomic inspection list through the text information, and jumps the mobile terminal display interface to the corresponding position.

[0096] Specifically, the inspector can manually operate by sliding gestures to query other adjacent atomic inspection orders in the current display interface or use zoom gestures to enlarge the target inspection order to observe multiple atomic inspection orders. In addition, to avoid manual errors by inspectors, sensing devices such as gravity sensors or gyroscopes can be used. For example, when the mobile phone is tilted at a certain angle relative to the horizontal plane, the content of the display interface can slide the displayable atomic inspection orders according to the tilt direction corresponding to the tilt angle, thereby eliminating the need for inspectors to manually operate the display interface of the mobile terminal. Further, voice positioning can be used to analyze the position of the inspection item that the inspector needs to locate through language-to-text conversion, and then the mobile terminal interface can be jumped to the position of the target inspection item to achieve the positioning function.

[0097] 103. Obtain the checkpoint images and inspection audio of the target store, extract the first inspection feature and the second inspection feature of the target store, input the inspection features into the industry model for fusion analysis and output the inspection results, and fill in the atomic inspection form according to the inspection results to fill in the target inspection form on the mobile terminal.

[0098] In an optional embodiment, the step of acquiring the inspection point image and inspection audio of the target store, extracting the first inspection feature and the second inspection feature of the target store, and inputting the inspection features into the industry model for fusion analysis and outputting the inspection results includes:

[0099] Generate an inspection plan including a target inspection list based on the target store information, collect offline inspection information and interview voice data at the target store according to the inspection plan, and collect image data corresponding to the inspection items through visual equipment;

[0100] The voice data and the image data are uploaded to the processing system, the voice data is converted into text to obtain text data, the visual processing system uses a neural network to identify the image data and extract the first inspection feature, and the text processing system uses natural language processing to identify the text data and extract the second inspection feature;

[0101] Industry data and inspection data are used as training sets to update the industry model. The first inspection feature and the second inspection feature are input into the industry model to obtain the evaluation results of the corresponding inspection items on the atomic inspection list. The evaluation results from different feature sources are classified and sorted using the veto system, and the inspection results are output.

[0102] In an optional embodiment, filling in the atomic inspection form according to the inspection result to fill in the target inspection form of the mobile terminal includes:

[0103] For the target inspection form, the audio recognition reporting items and visual recognition reporting items in each atomic inspection form are automatically filled in on the mobile terminal according to the inspection results;

[0104] The offline identification and reporting items of the target inspection form are manually filled in by referring to the instructions and positioning components, and the target inspection results of the target store are recorded according to the target inspection form.

[0105] Specifically, when actually conducting inspections of chain stores, inspectors are required to determine the inspected stores and the inspection order within the responsible area, and generate a reference inspection plan. In order to reduce the workload of inspectors, they can view the on-site conditions of the target stores through monitoring equipment or cameras before conducting offline inspections, and then automatically upload the captured on-site videos or images to the AI ​​visual processing system of the inspection platform. Specifically, a neural network can be used to perform feature processing on the collected images or videos. The results of the feature processing can then be further analyzed through the trained industry model to see whether there are corresponding plus or minus items in the images or videos for filling in on the inspection form. Specifically, the camera or monitor will collect image and video information at each checkpoint in the target store, such as stove hygiene, whether work clothes are worn, whether masks are worn, etc.

[0106] Furthermore, in addition to the video data or image data collected online, inspectors still need to go to the store to complete offline inspections. In addition to observing some images offline that cannot be captured by cameras or monitors (such as sanitary blind spots, sanitary conditions in areas blocked by equipment), inspectors are also required to interview employees and store managers of the target store based on the inspection items in the target inspection form. The interview content can be collected through a mobile recording component or other recording equipment and automatically uploaded to the AI ​​voice recognition system of the inspection platform for conversion into text data (such as how many employees there are in the store, store business hours, store cleaning time periods, store cleaning times, number of times kitchen utensils are cleaned, etc.). The recognizable text data will be further provided to the industry big model for analysis to obtain the results of evaluating each inspection item in the atomic inspection form, where the results of each inspection item include but are not limited to filled in content such as qualified, unqualified, plus points, minus points, etc.

[0107] Furthermore, in addition to audio and video processing in the cloud to reduce the workload of inspectors, there are still some inspection items in the target stores that cannot be judged by audio and video but need to be judged by manual prior experience. For example, checkpoint areas that cannot be captured by monitoring, especially situations where the equipment needs to be moved to see clearly. Inspectors can check by reaching out and touching. In addition to hygiene issues, there are also issues such as whether the production process is strictly carried out in accordance with SOP requirements. In addition to confirming this inspection item through voice-to-text interviews, inspectors still need to conduct actual visits to the production end to obtain accurate conclusions. Therefore, in order to further improve the efficiency of inspections, the corresponding content of the above-mentioned special inspection items can also be configured in the atomic inspection form to ensure that inspectors can fill in this part of the content when using the mobile terminal, and cooperate with the cloud platform to process image data and audio data to quickly fill in the inspection form, thereby improving inspection efficiency.

[0108] The present invention proposes a mobile terminal store inspection method, system, device and medium for audio and video cloud processing, wherein the method includes: obtaining the target inspection demand to formulate the first standard of chain stores, obtaining the target store information to formulate the second standard, grading the inspection items according to the first standard and the second standard, and outputting the target inspection list; adapting the target inspection list to the mobile terminal display interface and splitting it into atomic inspection lists, configuring the atomic inspection list with a query instruction and a positioning component to traverse the target inspection list; obtaining the inspection point image and inspection audio of the target store, extracting the first inspection feature and the second inspection feature of the target store, inputting the inspection feature into the industry model for fusion analysis and outputting the inspection result, and filling the atomic inspection list according to the inspection structure to fill in the target inspection list. The analysis results of the inspection items and inspection data processed by the cloud can help the inspectors to quickly complete the inspection process of the target stores, and at the same time introduce the operability of the mobile terminal display, which not only improves the inspection efficiency but also improves the accuracy of the inspection results, ensuring that the inspection results can reflect the actual store situation, especially in the field of chain store inspection, which has a high application value.

[0109] like Figure 3 As shown, the second aspect of the present invention discloses a mobile store inspection system for audio and video cloud processing, the system comprising:

[0110] Inspection form generation module, used to obtain the target inspection requirements to formulate the first standard of chain stores, obtain the target store information to formulate the second standard, classify the inspection items according to the first standard and the second standard, and output the target inspection form;

[0111] A mobile terminal configuration module is used to adapt the target inspection list to the mobile terminal display interface and split it into atomic inspection lists, and configure the atomic inspection list with a query instruction and a positioning component to traverse the target inspection list;

[0112] The inspection form filling module is used to obtain the inspection point images and inspection audio of the target store, extract the first inspection features and second inspection features of the target store, input the inspection features into the industry model for fusion analysis and output the inspection results, and fill in the atomic inspection form according to the inspection results to fill in the target inspection form on the mobile terminal.

[0113] In an optional embodiment, the inspection form generation module includes:

[0114] A standard generation module is used to determine the target inspection requirements based on historical inspection data and inspection update rules, and to determine the standard inspection items of each cluster store based on the cluster classification of the chain stores to generate the first standard;

[0115] Determine, according to the target store information, special inspection items that distinguish the target store from the cluster stores, and generate a second standard with the special inspection items;

[0116] The first standard and the second standard are configured as an inspection template and saved in a database, and the inspection template is updated in real time.

[0117] In an optional embodiment, the inspection form generation module includes:

[0118] An inspection list generation submodule, used for performing hierarchical processing on the inspection items of the first standard and the second standard by using a classification model, wherein the hierarchical processing inspection items include inspection categories and inspection items, and the inspection items are classified into the inspection categories;

[0119] According to the hierarchical processing, the inspection categories and the inspection items within the inspection categories are prioritized, and a first display order template is obtained to generate a target inspection list.

[0120] In an optional embodiment, the mobile terminal configuration module includes:

[0121] An inspection form splitting module is used to perform a first weight allocation on the inspection categories and inspection items in the target inspection form according to the inspection form reporting type, and allocate a sliding operation distance of the target reporting item in the target inspection form from the mobile terminal display interface according to the first weight allocation, wherein the reporting type includes an audio recognition reporting item, a visual recognition reporting item, and an offline recognition reporting item;

[0122] A second weight is added to the target inspection list assigned with a first weight based on the target store information and the inspector's experience information, and the inspection items in the target inspection list are split into atomic inspection lists based on the first weight and the second weight. The atomic inspection list is configured on the mobile display interface to assign display color and display order based on the first weight and the second weight.

[0123] In an optional embodiment, the mobile terminal configuration module includes:

[0124] A mobile terminal configuration submodule is used for the mobile terminal to fill the first atomic inspection list in the display interface, and to set the second atomic inspection list adjacent to the first atomic inspection list in sequence according to the first weight and the second weight;

[0125] The mobile terminal configures a sliding gesture, a zooming gesture and a sensing device for the second atomic inspection sheet to view the second atomic inspection sheet adjacent to the first atomic inspection sheet, and the sensing device scrolls the display content of the display interface according to the tilt state of the mobile terminal device;

[0126] The mobile terminal obtains voice information and converts it into text information, determines the position of the second atomic inspection list through the text information, and jumps the mobile terminal display interface to the corresponding position.

[0127] In an optional embodiment, the inspection form filling module includes:

[0128] The feature analysis module is used to generate an inspection plan including a target inspection list based on the target store information, collect offline inspection information and interview voice data at the target store according to the inspection plan, and collect image data corresponding to the inspection items through visual equipment;

[0129] The voice data and the image data are uploaded to the processing system, the voice data is converted into text to obtain text data, the visual processing system uses a neural network to identify the image data and extract the first inspection feature, and the text processing system uses natural language processing to identify the text data and extract the second inspection feature;

[0130] Industry data and inspection data are used as training sets to update the industry model. The first inspection feature and the second inspection feature are input into the industry model to obtain the evaluation results of the corresponding inspection items on the atomic inspection list. The evaluation results from different feature sources are classified and sorted using the veto system, and the inspection results are output.

[0131] In an optional embodiment, the inspection form filling module includes:

[0132] The inspection form filling submodule is used to automatically fill in the audio recognition filling items and visual recognition filling items in each atomic inspection form on the mobile terminal according to the inspection results of the target inspection form;

[0133] The offline identification and reporting items of the target inspection form are manually reported by consulting the instructions and positioning components, and the target inspection results of the target store are recorded according to the target inspection form. The third aspect of the present invention discloses a mobile terminal store inspection device for audio and video cloud processing, including:

[0134] at least one processor, and

[0135] a memory communicatively connected to the at least one processor; wherein,

[0136] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the mobile store inspection method for audio and video cloud processing as described in any one of the first aspects of the present invention.

[0137] The computer device may be a terminal, and the computer device includes a processor, a memory, a network interface, a display screen, and an input device connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a mobile terminal store inspection method for audio and video cloud processing is implemented. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covering the display screen, or a button, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0138] The fourth aspect of the present invention discloses a computer-readable storage medium, which stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the mobile store inspection method for audio and video cloud processing as described in any one of the items disclosed in the first aspect of the present invention.

[0139] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing related hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it may include the processes of the embodiments of the mobile store inspection method processed by the audio and video cloud as described above. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct RAMbus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0140] Alternatively, if the above modules of the present invention are implemented in the form of software function modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the relevant technology. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, terminal, or network device, etc.) to execute all or part of the methods of each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, RAM, ROM, magnetic disks or optical disks.

[0141] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A mobile terminal store inspection method for audio and video cloud processing, characterized in that: The method comprises: Obtain the target inspection requirements to formulate the first standard for chain stores, obtain the target store information to formulate the second standard, classify the inspection items according to the first standard and the second standard, and output the target inspection form; The target inspection list is adapted to the mobile terminal display interface and split into atomic inspection lists. The atomic inspection lists are configured with reference instructions and positioning components to traverse the target inspection list. Obtain the checkpoint images and inspection audio of the target store, extract the first inspection features and second inspection features of the target store, input the inspection features into the industry model for fusion analysis to output the inspection results, and fill in the atomic inspection form according to the inspection results to fill in the target inspection form on the mobile terminal.

2. The mobile terminal store inspection method for audio and video cloud processing according to claim 1 is characterized in that: The first standard of obtaining the target inspection requirements and formulating the chain stores, and the second standard of obtaining the target store information and formulating the chain stores, include: Determine the target inspection requirements based on historical inspection data and inspection update rules, and determine the standard inspection items of each cluster store based on the cluster classification of chain stores to generate the first standard; Determine, according to the target store information, special inspection items that distinguish the target store from the cluster stores, and generate a second standard with the special inspection items; The first standard and the second standard are configured as an inspection template and saved in a database, and the inspection template is updated in real time.

3. The mobile terminal store inspection method of audio and video cloud processing according to claim 2 is characterized in that: The step of grading the inspection items according to the first standard and the second standard and outputting the target inspection list includes: The classification model is used to perform hierarchical processing on the inspection items of the first standard and the second standard, wherein the hierarchical processing inspection items include inspection categories and inspection items, and the inspection items are classified into the inspection categories; According to the hierarchical processing, the inspection categories and the inspection items within the inspection categories are prioritized, and a first display order template is obtained to generate a target inspection list.

4. The mobile terminal store inspection method of audio and video cloud processing according to claim 3 is characterized in that: The step of adapting the target inspection list to the mobile terminal display interface and splitting it into atomic inspection lists includes: Performing a first weight allocation on the inspection category and the inspection item in the target inspection form according to the inspection form reporting type, and allocating a sliding operation distance of the target reporting item in the target inspection form from the mobile terminal display interface according to the first weight allocation, wherein the reporting type includes an audio recognition reporting item, a visual recognition reporting item, and an offline recognition reporting item; A second weight is added to the target inspection list assigned with a first weight based on the target store information and the inspector's experience information, and the inspection items in the target inspection list are split into atomic inspection lists based on the first weight and the second weight. The atomic inspection list is configured on the mobile display interface to assign display color and display order based on the first weight and the second weight.

5. The mobile terminal store inspection method of audio and video cloud processing according to claim 4 is characterized in that: The configuration of the atomic inspection list with a reference instruction and a positioning component to traverse the target inspection list includes: The mobile terminal fills the first atomic inspection list in the display interface, and sets the second atomic inspection list adjacent to the first atomic inspection list in sequence according to the first weight and the second weight; The mobile terminal configures a sliding gesture, a zooming gesture and a sensing device for the second atomic inspection sheet to view the second atomic inspection sheet adjacent to the first atomic inspection sheet, and the sensing device scrolls the display content of the display interface according to the tilt state of the mobile terminal device; The mobile terminal obtains voice information and converts it into text information, determines the position of the second atomic inspection list through the text information, and jumps the mobile terminal display interface to the corresponding position.

6. The mobile terminal store inspection method of audio and video cloud processing according to claim 5 is characterized in that: The method of obtaining the inspection point image and inspection audio of the target store, extracting the first inspection feature and the second inspection feature of the target store, and inputting the inspection features into the industry model for fusion analysis and outputting the inspection results includes: Generate an inspection plan including a target inspection list based on the target store information, collect offline inspection information and interview voice data at the target store according to the inspection plan, and collect image data corresponding to the inspection items through visual equipment; The voice data and the image data are uploaded to the processing system, the voice data is converted into text to obtain text data, the visual processing system uses a neural network to identify the image data and extract the first inspection feature, and the text processing system uses natural language processing to identify the text data and extract the second inspection feature; Industry data and inspection data are used as training sets to update the industry model. The first inspection feature and the second inspection feature are input into the industry model to obtain the evaluation results of the corresponding inspection items on the atomic inspection list. The evaluation results from different feature sources are classified and sorted using the veto system, and the inspection results are output.

7. The mobile terminal store inspection method of audio and video cloud processing according to claim 6 is characterized in that: The filling of the atomic inspection form according to the inspection result to fill in the target inspection form of the mobile terminal includes: For the target inspection form, the audio recognition reporting items and visual recognition reporting items in each atomic inspection form are automatically filled in on the mobile terminal according to the inspection results; The offline identification and reporting items of the target inspection form are manually filled in by referring to the instructions and positioning components, and the target inspection results of the target store are recorded according to the target inspection form.

8. A mobile store inspection system with audio and video cloud processing, characterized in that: The system comprises: Inspection form generation module, used to obtain the target inspection requirements to formulate the first standard of chain stores, obtain the target store information to formulate the second standard, classify the inspection items according to the first standard and the second standard, and output the target inspection form; A mobile terminal configuration module is used to adapt the target inspection list to the mobile terminal display interface and split it into atomic inspection lists, and configure the atomic inspection list with a query instruction and a positioning component to traverse the target inspection list; The inspection form filling module is used to obtain the inspection point images and inspection audio of the target store, extract the first inspection features and second inspection features of the target store, input the inspection features into the industry model for fusion analysis and output the inspection results, and fill in the atomic inspection form according to the inspection results to fill in the target inspection form on the mobile terminal.

9. A mobile store inspection device with audio and video cloud processing, characterized in that: include: at least one processor, and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the mobile store inspection method for audio and video cloud processing as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the mobile store inspection method for audio and video cloud processing as described in any one of claims 1 to 7.

Images